LCP SDK caches captive portal HTML as CRL when device is offline

[marcommobile]

Description

We have identified a potential issue with the mobile version of the app on Android and suspect the same issue may also exist on iOS.

Problem

When the app needs to update the certificate/CRL used by the Readium LCP encryption process, a user may be connected to Wi-Fi but have no actual internet access – for example, when connected to an airplane's Wi-Fi network.

In this situation, the network may redirect the HTTP request to a captive portal (e.g. an HTML page asking the user to purchase Wi-Fi).

Our current theory is that the Readium LCP SDK receives this HTML page instead of the expected CRL, but treats it as a valid response because the HTTP status code is 200 OK.

The SDK then caches this invalid response as the CRL.

Suspected problematic code

The following Android code appears to be responsible for fetching the CRL:

Kotlin
private suspend fun fetch(): String {
    val url = "http://crl.edrlab.telesec.de/rl/EDRLab_CA.crl"
    val data = network.fetch(url, NetworkService.Method.GET)
        .getOrElse { throw LcpException(LcpError.CrlFetching) }

    return if (Build.VERSION.SDK_INT >= Build.VERSION_CODES.O) {
        "-----BEGIN X509 CRL-----${Base64.getEncoder().encodeToString(data)}-----END X509 CRL-----"
    } else {
        "-----BEGIN X509 CRL-----${android.util.Base64.encodeToString(
            data,
            android.util.Base64.DEFAULT
        )}-----END X509 CRL-----"
    }
}

The underlying network implementation appears to consider any response below HTTP 400 to be successful:

Kotlin
suspend fun fetch(
    url: String,
    method: Method = Method.GET,
    parameters: URLParameters = emptyMap(),
    timeout: Duration? = null,
    headers: Map<String, String> = emptyMap(),
): Try<ByteArray, NetworkException> =
    withContext(Dispatchers.IO) {
        try {
            @Suppress("NAME_SHADOWING")
            val url = URL(
                Uri.parse(url).buildUpon().appendQueryParameters(parameters).build().toString()
            )

            val connection = url.openConnection() as HttpURLConnection
            connection.requestMethod = method.value
            if (timeout != null) {
                connection.connectTimeout = timeout.inWholeMilliseconds.toInt()
            }
            connection.appendRequestHeaders(headers)

            val status = connection.responseCode
            if (status >= 400) {
                Try.failure(NetworkException(status))
            } else {
                Try.success(connection.inputStream.readBytes())
            }
        } catch (e: Exception) {
            Timber.e(e)
            Try.failure(NetworkException(status = null, cause = e))
        }
    }

Suspected root cause

It appears that the SDK only validates the HTTP status code and does not verify that the response actually contains a valid X.509 CRL before processing and caching it.

For example:

1. The device is connected to airplane Wi-Fi.
2. The device has no internet access.
3. The LCP SDK attempts to download the CRL.
4. The request is intercepted by the Wi-Fi provider's captive portal.
5. The server returns an HTML page with HTTP 200 OK.
6. The SDK considers the response successful.
7. The HTML content is encoded and treated as a CRL.
8. The invalid CRL is cached.

Impact

The most problematic part is that the invalid response appears to remain cached for 7 days.

As a result, even after the user reconnects to a normal internet connection, the SDK continues using the corrupted cached CRL until the cache expires.

This can effectively leave the user unable to use affected LCP-protected content for several days.

Expected behavior

The SDK should validate the downloaded data as a valid CRL before caching it.

If the response is not a valid CRL – for example, if it contains HTML from a captive portal – the SDK should:

• Reject the response.
• Avoid caching the invalid data.
• Keep the existing valid CRL, if one is available.
• Retry the CRL update when a valid internet connection becomes available.
• Ideally, handle captive portal responses gracefully.

Questions / Investigation

• Is this theory correct?
• Does the current LCP SDK validate the CRL contents before caching them?
• Could this issue also affect the iOS implementation?
• What would be the recommended fix/workaround?
• Can the SDK validate the downloaded data as an actual X.509 CRL before storing it in the cache?
• Is there a way to invalidate/remove an already corrupted cached CRL so affected users do not have to wait for the 7-day cache expiration?

[mickael-menu]

Thank you for reporting this issue @marcommobile, it should be fixed in #833, which will also recover from already cached poisoned CRLs.

[marcommobile]

Awesome. Thank you, @mickael-menu.

[marcommobile]

@mickael-menu, just a quick question: does the same issue exist in the iOS version of the SDK?

[mickael-menu]

Yes, it will be fixed in the next 4.0 alpha release: readium/swift-toolkit#894