diff --git a/device/cheetah/IMS-SHANNON.md b/device/cheetah/IMS-SHANNON.md new file mode 100644 index 000000000..56444429d --- /dev/null +++ b/device/cheetah/IMS-SHANNON.md @@ -0,0 +1,377 @@ +# IMS-SHANNON: how stock's IMS service drives the CP's IMS stack + +Status: research note, 2026-11-02 (offline). Source: Shannon's IMS service +APK from the cheetah vendor tree (`system_ext/priv-app/ShannonIms`, +versionName `000000.000000.b9d325b79`, sha256 prefix `dd3889d95e280717`), +decompiled with jadx 1.5.1 on the laptop. Nothing of it is in the +repository; this note gives class and method names, request IDs, field names +and offsets only. Consumer: `packages/omamobile-modem-sit` (`lib/ims.py`, +`lib/bringup.py`, `lib/data_call.py`). + +Question this answers: why cheetah's CP answers our ADD_PDN_INFO on +Tracfone/Straight Talk on Verizon (311480) with IND_AIMS_REGISTRATION state +1, features 0, status 5 and never registers, while the same sequence +registers on T-Mobile. + +**Short answer.** Status 5 is `IMS_SERVICE_REGISTER_FAIL`, the permanent +registration failure. Stock never puts the CP in that position on this +path: it **never sends AIMS_ADD_PDN_INFO for IMS without a P-CSCF**. When +the IMS PDN's link properties carry none and no ISIM P-CSCF resolved, stock +stops ("stop registration for invalid pcscf", reason `NO_PCSCF`) and tears +the PDN down; `BLOCK_PDN_BY_INVALID_PCSCF` (0 for Verizon) only decides +whether the PDN is also blocked until a power cycle. ADD_PDN_INFO has no +FQDN field and no "P-CSCF from SIM" flag: the list is binary IPv4 and IPv6 +addresses only, so the CP doesn't resolve or discover anything itself. On +Verizon stock's P-CSCF comes from the IMS PDN's PCO (the RIL's data-call +response), or from the ISIM's EF_PCSCF name resolved **on the default +network** (the internet APN without Wi-Fi). Our IMS bearer request differs +from stock's for Verizon's IPv4v6 IMS APN; see section 7. + +## 1. Where things live + +| Role | Class (package `com.shannon.imsservice`) | +| --- | --- | +| Stack start/stop, hidden menu, ADD_PDN order | `conn.StackControl` (`startCpModules`, `startCpImsModules`, `sendHiddenMenu`, `sendPdnInfo`, `isRequiredOmaDmHiddenMenu`, `updateNtpTime`) | +| Request encoders | `network.adaptor.RegistrationAdaptor` (`sendStackStartReq`, `sendHiddenMenu`, `sendHiddenMenuCpImsConfiguration`, `sendHiddenMenuOmaDm`, `sendAddPdnInfo`, `sendDelPdnInfo`, `sendDeregistration`, `sendSetHiddenMenuItem`, `sendImsTestModeState`, `sendSetPdnEstStatus`, `sendRegistrationState`, `updateNtpTime`) | +| Hidden-menu TLVs | `network.adaptor.HiddenMenuBuilder` (and its enum `CpImsConfiguration`) | +| Wire layouts | `network.adaptor.data.RilReq*` / `RilInd*` (`getBytes`, `update`) | +| P-CSCF selection | `conn.ApnEntity.updatePcscfAddresses`, its network callback `handleNetworkAvailable`; `conn.ImsConnectivityMgr.getPcscfAddressFromSim` / `addPcscfFromSim` / `getHostNameFromSim`; `conn.DefaultNetworkCallback.onAvailable` | +| Registration indication | `network.adaptor.data.RilIndRegistration` (`RegState`, `RegFeature`), `conn.RegistrationReason` | +| Per-carrier policy | assets `sim_configurations/sim_configuration_311.xml`, `sim_operator_list.xml`, `operator_config.xml`; defaults `config.configuration.ImsSimConfigDefaults` | +| Message IDs | `network.oem.helper.ImsRilHelper.RilMessageId` (RIL-OEM numbers; the RIL maps them to SIT IDs) | + +The service talks to the RIL with RIL-OEM message numbers, not SIT IDs. The +payloads are the RIL-OEM structures below (little-endian, `DataWriter`). +Our daemon sends these structures as the SIT payloads of the IDs in +`lib/sit_names.py`; for HIDDEN_MENU and ADD_PDN_INFO that is proven on the +phone (T-Mobile registered). For the others the pass-through is assumed, +not verified in the RIL. + +| RIL-OEM name (number) | SIT ID / `sit_names.py` | +| --- | --- | +| `RILC_REQ_AIMS_STACK_START_REQ` (137) | 0x0d39 AIMS_STACK_START_REQ | +| `RILC_REQ_AIMS_XCAPM_START_REQ` (139) | 0x0d3b AIMS_XCAPM_START_REQ | +| `RILC_REQ_AIMS_SET_FRAME_TIME` (108) | 0x0d0b AIMS_SET_FRAME_TIME | +| `RILC_REQ_AIMS_HIDDEN_MENU` (104) | 0x0d07 AIMS_HIDDEN_MENU | +| `RILC_REQ_AIMS_ADD_PDN_INFO` (105) | 0x0d08 AIMS_ADD_PDN_INFO | +| `RILC_REQ_AIMS_DEL_PDN_INFO` (136) | 0x0d38 AIMS_DEL_PDN_INFO | +| `RILC_REQ_AIMS_SET_HIDDEN_MENU_ITEM` (146) | 0x0d36 | +| `RILC_REQ_AIMS_SET_PDN_EST_STATUS` (145) | 0x0d71 | +| `RILC_REQ_AIMS_SET_SELF_CAPABILITY` (129) | 0x0d27 | +| `RILC_REQ_AIMS_COMMON_IMS_SVC_INFO` (161) | 0x0d88 | +| `RILC_REQ_SET_IMS_TEST_MODE` (38) | 0x0614 SET_IMS_TEST_MODE | +| `RILC_REQ_SET_IMS_REGISTRATION_STATE` (48) | not in `sit_names.py` | +| `RILC_REQ_MISC_SET_PREFERRED_CALL_CAPA` (20) / `SET_VOICE_OPERATION` (45) | 0x092f / 0x091a | +| `RILC_UNSOL_AIMS_REGISTRATION` (3012) | 0x0d05 IND_AIMS_REGISTRATION | +| `RILC_UNSOL_CP_CARRIER_CONFIG_CHANGED` (9000) | the 0x4601 status (u32 status, u32 cause; status 1 releases the stack start) | + +## 2. The sequence, stack start to registered (cellular) + +1. SIM loaded, carrier config status received (`onCarrierConfigManifestStatusReceived`) + → `StackControl.startCpModules(slot, simReady)`: + - **AIMS_STACK_STOP_REQ** first if a previous stop failed, 50 ms; + - **AIMS_STACK_START_REQ** `[u8 sim status (1 ready), u8 stack type (slot + 1)]` = `01 01`; + - **AIMS_XCAPM_START_REQ** (same two bytes), when the XCAP stack may start; + - **AIMS_SET_FRAME_TIME** (`updateNtpTime`): `[u8 call id 0, u8 operation 2, u64 time]`, the time being the AP's wall clock in milliseconds since 1970. +2. The service asks Android for the IMS network (APN `IMS` on Verizon, the + attach PDN). On `handleNetworkAvailable`: `updatePcscfAddresses(linkProperties)`. + Link properties' P-CSCF list empty → the list resolved from the ISIM + (section 4); still empty → **stop: no ADD_PDN**, `NO_PCSCF`, PDN + disconnected. Otherwise → `updateStartImsStack`, then + `onApnEnabled` → `StackControl.sendPdnInfo`. +3. `sendPdnInfo`, first IMS PDN after a stack start: + - **AIMS_HIDDEN_MENU** part 1, more-expected 1 (section 5); + - **AIMS_HIDDEN_MENU** CP IMS configuration segments, only when the + carrier config enables AOSP IMS configuration (section 6); + - **AIMS_HIDDEN_MENU** part 2, more-expected 0: the video media + description (`MediaConfig.getMediaInfo`, else a built-in default); + - then `RegistrationAdaptor.sendAddPdnInfo`, which first sends the + **OMA-DM AIMS_HIDDEN_MENU** (more-expected 0) once after SIM load, when + no ADD_PDN has been sent yet (`isRequiredOmaDmHiddenMenu`, static + `setOmaDm`); + - **AIMS_ADD_PDN_INFO** (section 3). +4. The CP registers and sends IND_AIMS_REGISTRATION (section 8). On state 2 + the service sends SET_IMS_REGISTRATION_STATE `[u32 1, u32 RAT]` (a + report to the RIL, not a precondition) and, with RCS, SET_SELF_CAPABILITY. +5. Later, only on events: SET_HIDDEN_MENU_ITEM 519 (the user agent, when it + changes), SET_PDN_EST_STATUS `[u8 status]` (2 when the IMS registration + timer expires or Verizon's throttling outlasts it; 1 for AT&T's + not-subscribed), COMMON_IMS_SVC_INFO (Wi-Fi calling button, VoLTE switch), + SET_IMS_TEST_MODE `[u32]` (only when the `IMS_TEST_MODE` policy changes; + default 0, never sent at start-up). + +Our sequence (0.3.19) was: STACK_START `01 01` → SETUP_DATA_CALL (IMS, IPv6) → +HIDDEN_MENU (one segment, more-expected 0) → ADD_PDN_INFO (empty list) → +status 5. Missing against stock: XCAPM_START, SET_FRAME_TIME, the OMA-DM +menu, hidden-menu part 2, and above all a P-CSCF. + +## 3. AIMS_ADD_PDN_INFO (0x0d08) layout + +`RilReqAddPdnInfo.getBytes`, filled by `RegistrationAdaptor.sendAddPdnInfo`. +Offsets for a cellular PDN with `n4` IPv4 and `n6` IPv6 P-CSCF addresses +(the list position makes everything after it move). + +| Offset | Size | Field | Stock value on cellular IMS | +| --- | --- | --- | --- | +| 0 | 1 | PDN type | `ApnNetType.getApnCategory`: IMS 2 | +| 1 | 1 | IP type | `ApnEntity.getIpType`: bit 0 IPv4, bit 1 IPv6 (3 for both) | +| 2 | 4 | PDN IPv4 address | 0.0.0.0 when none | +| 6 | 16 | PDN IPv6 address | :: when none | +| 22 | 1 | HD internet enabled | 17 | +| 23 | 1 | operator ID | 0 | +| 24 | 1 | registration style | `getRegStyle`: 0 normal, 1 for handover/re-registration | +| 25 | 1 | PDN new | 1 new, 0 for handover/re-registration | +| 26 | 1 | VoLTE auto-config | 0 | +| 27 | 1 | RCS auto-config | 0 | +| 28 | 2 | offering type | `getOfferingType`: capability bits (VoLTE 1, SMS-IP 2, RCS 4, PSVT 8, RTT 16, mobile data 32, Wi-Fi data 64, ViCE 256) | +| 30 | 1 | registration type | `ApnNetType.getRegType` (IMS 0; 3 with RCS on) | +| 31 | 1 | init type | 0, 1 handover, 2 re-registration | +| 32 | 1 | IMS net type | transport: 1 cellular, 2 Wi-Fi | +| 33 | 6 | Wi-Fi MAC | zero on cellular | +| 39 | 1 | IPv4 P-CSCF count `n4` | at most 8 | +| 40 | 4·n4 | IPv4 P-CSCF addresses | | +| 40+4n4 | 1 | IPv6 P-CSCF count `n6` | at most 8 | +| 41+4n4 | 16·n6 | IPv6 P-CSCF addresses | | +| then | 4, 16, 4, 16 | primary DNS IPv4, IPv6; secondary DNS IPv4, IPv6 | from the link properties | +| then | 23 | last access network, two 8-byte timestamps, TAC (u16), cell ID (u32) | Wi-Fi only; zero on cellular | +| then | 4 | remaining emergency registration timer | 0 | +| then | 10 | network MCC/MNC digits (6), country code (2), MTU (2) | Wi-Fi only; zero on cellular | +| then | 4, 2, 4 | RCS capability mask, vendor capability mask, extended RCS mask | 0 without RCS | +| then | 1 + 32 | VoWiFi E911 AID length and text | empty | +| then | 2 | additional-field length | 12 on cellular | +| then | 500 | additional fields, fixed buffer: TLV `u16 tag 1, u16 len 8, u64 SIP delegate capability`; on Wi-Fi also band (tag 2), cell ID (4), TAC (5), interface index (6) | | + +There is **no FQDN, domain or "from SIM" field**. Every P-CSCF is a binary +address the AP resolved. Compared with our `add_pdn_payload` (0.3.19): the +layout matches field for field; differences were IP type fixed at 2 (IPv6 +only, IPv4 address and IPv4 DNS always zero), at most 3 IPv6 P-CSCFs (stock 8) +and, by configuration, an empty list (stock: never). 0.3.20 announces an +IPv4v6 bearer with both families as stock does (section 9). + +AIMS_DEL_PDN_INFO (0x0d38): `[u8 IMS net type, u8 PDN type, u8 reason]`. +AIMS_DEREGISTRATION (0x0d06): `[u8 PDN type, u16 feature 126, u8 cause 0]`. + +## 4. P-CSCF handling in stock + +`ApnEntity.updatePcscfAddresses(LinkProperties)`, for IMS, emergency and +RCS PDNs: + +1. `linkProperties.getPcscfServers()`: the P-CSCF list Android's data stack + got from the RIL's data-call response, which the CP fills from the PDN's + PCO. +2. If empty: `ImsConnectivityMgr.mPcscfAddressFromSim`. That list is built by + `getHostNameFromSim` (`TelephonyManager.getIsimPcscf()`: the ISIM's + EF_PCSCF, non-alphanumeric characters stripped) and + `getPcscfAddressFromSim`, which resolves each name with + `InetAddress.getByName` on its own thread. That runs when the **default + network** becomes available (`DefaultNetworkCallback.onAvailable`), so the + name is resolved by the default network's DNS: the internet APN's DNS + servers when there is no Wi-Fi, never the IMS PDN's. `getByName` returns + one address per name, IPv4 or IPv6, whichever the resolver puts first. +3. Split into IPv4 and IPv6 lists by address family. Both empty → return + false → `handleNetworkAvailable` logs "stop registration for invalid + pcscf", sets `DeregiImsReasonInfo.NO_PCSCF` and disconnects; with + `BLOCK_PDN_BY_INVALID_PCSCF` = 1 it also blocks the PDN + (`blockPdnByInvalidPcscfIfNeeded`, `RETRY_COUNT_FOR_INVALID_PCSCF`). + Verizon's 311480 entry sets it to 0, so stock just waits for the next PDN + or default-network event; it still sends no ADD_PDN. + +So for this SIM stock registers only if (a) the IMS PDN's PCO carries a +P-CSCF, or (b) `pcscf1.anyims.vzims.com` resolves through the internet APN's +DNS. Which of the two it is on this line can only be read from a stock +logcat (section 10, step 0). + +## 5. AIMS_HIDDEN_MENU (0x0d07) and the provisioning requests + +Request (`RilReqHiddenMenu`): `[u8 more-expected, u16 data length, data]`. +Data is a list of items (`HiddenMenuBuilder`): `u16 tag` then either a u32 +integer (integers of -1 are skipped), or a string/byte string followed by +`##` (0x23 0x23). Segments at most 900 data bytes. + +Part 1 (`RegistrationAdaptor.sendHiddenMenu`, more-expected 1): + +| Tag | Value | +| --- | --- | +| 523, 527, 529, 531, 533 | 30 each (SIP timers A, E, G, I, K) | +| 772 / 773 / 774 / 775 / 776 | manufacturer / model / build ID / product name / OS version (`DeviceInfoUtil`) | +| 519 | the user agent (`SIP_USER_AGENT` template; Verizon `#MANUFACTURE#---#MODEL#---#BUILD#`) | +| 1044, 1045 | geolocation PIDF for VoWiFi / VoLTE emergency calls (Verizon: 1045 = 1) | +| 1080, 1081, 1082 | EPS-fallback silent-retry error code, VoLTE/VoWiFi silent-retry policy and codes (default 0) | +| 1084 | RTT supported | + +Part 2 (more-expected 0): the video media description (`SetVideoMediaInfo`: +bandwidth, CVO, ECN, H.264/H.265 fmtp lines), a binary structure, not TLVs. +It configures video calls, not registration. Our single segment with +more-expected 0 registered on T-Mobile, so the CP accepts the menu without it. + +OMA-DM menu (`sendHiddenMenuOmaDm`, more-expected 0, once before the first +ADD_PDN after SIM load, on every carrier): tag **777 = 1** (is-OMA-DM-update), +then the provisioned `OmaDmProvisionedDataType` values: 778 domain name +(skipped while it is the default "IMS"), 779 T1, 780 TF, 781 T2, 782 AMR-NB +mode set, 783 AMR-WB mode set, 784 session expiry, 785 Min-SE, 786 +cancellation timer ("HVOLTE TVZW"), 787 Tdelay, 788 silent redial. Defaults +are -1 (skipped) for the integers and empty strings for 782/783 (sent as +`tag ##`). With nothing provisioned the menu is `777=1` plus the two empty +AMR items. + +SET_HIDDEN_MENU_ITEM (0x0d36): `[u16 tag, u16 length, data padded to 100]`. +GET_HIDDEN_MENU_ITEM (0x0d37): the tag. + +None of these carry a P-CSCF, domain, IMPI/IMPU or transport. The CP's IMS +stack reads IMPI, IMPU and the home domain from the ISIM itself (the service +only checks the ISIM domain isn't the placeholder "IMS", +`isValidIsimDomain`), and its SIP settings come from the CP's carrier +profile (the 0x4600 manifest), unless section 6 applies. + +"VoLTE enabled", "SMS over IMS enabled": no request carries them at +registration. They become the offering type in ADD_PDN_INFO (VoLTE 1, +SMS-IP 2; `CapabilityManager`), and later SET_SELF_CAPABILITY +(`[u8 version 1, u16 feature, u32 mask, u32 extended mask, u64 delegate mask]`). + +## 6. Carrier config: what reaches the CP + +`sim_configuration_311.xml`, MNC 480 (Verizon; Tracfone-VZW matches it by +MCC/MNC, no MVNO override except Cellcom's GID). Keys there that change what +the CP sees or whether stock registers: + +| Key | Verizon value | Effect | +| --- | --- | --- | +| `BLOCK_PDN_BY_INVALID_PCSCF` | 0 | no PDN block after a P-CSCF-less PDN; still no ADD_PDN | +| `SIP_USER_AGENT` | `#MANUFACTURE#---#MODEL#---#BUILD#` | hidden-menu tag 519 | +| `GEOLOCATION_PIDF_FOR_VOLTE_EMERGENCY_CALL` | 1 | tag 1045 | +| `IMS_SINGLE_REGISTRATION_REQUIRED` | 1 | RCS on the IMS registration (RCS off by default) | +| `SMS_FORMAT` | 1 (3GPP) | SMS over IMS format | +| `LTE_VOPS_POLICY`, `USE_VOPS_STATUS_ROAM` | 1, 0 | whether stock requests the IMS PDN without VoPS | +| `IMS_PDN_ACTIVATION_TIMEOUT`, `NR_TIMER_WAIT_IMS_REGISTRATION` | 120, 120 | timers that end in SET_PDN_EST_STATUS 2 | +| `ALLOWED_RAT` | LTE, LTE_CA, NR, IWLAN | | +| `DATA_CALL_PERMANENT_FAILURE_FOR_IMS` | causes 0x8, 0x1B-0x1D, 0x20, 0x23, 0x32, 0x33 | PDN causes that stop IMS on cellular | + +The CP IMS configuration menu (`HiddenMenuBuilder.CpImsConfiguration`: SIP +port tag 10, registration expiry 11, retry timers 20/21, T1/T2/T4 534-536, +timers B-J, preferred transport 1049, IPsec 15/16 and 1031, phone-context +1025, URI type 1026, MTUs 1028/1029, GRUU 1046, reg-event 1047/1048, codecs +258 and 1060-1062, ...) is sent only when +`CarrierConfigManagerReceiver.isSupportAospImsConfiguration` holds: the +carrier config key `vendor.ims.carrier_config_use_aosp_ims_config_bool` (or a +persist property) and `UPDATE_IMSCONFIG_BY_CARRIERCONFIG`. Values come from +the `vendor.ims.*`/AOSP IMS keys. Whether Verizon's CarrierSettings sets the +key wasn't checked here (the decoded CarrierSettings in +MODEM-PROVIDER-PROFILES.md is the place). Without it the CP uses its own +carrier profile for all of these, which is the profile the 0x4600 manifest +selects (`us_tracfone`). + +## 7. Status 5 and the other registration values + +`RilIndRegistration.update`: `u8 state, u8 feature, u32 feature mask, u16 +status, u16 URI length (≤1000), u16 reason length (≤256), URI + reason text, +[u32 feature mask ext], [u64 SIP delegate mask], [u16 feature ext]`. Our +`record_registration` reads the same fields. + +States (`RegState`): 1 VoLTE not registered, 2 VoLTE registered, 3 VoLTE +limited, 4 E911 not registered, 5 E911 registered, 6 RCS not registered, 7 RCS +registered, 8 E911 pseudo-registered. Features (`RegFeature`): 0 unknown, 1 +VoLTE, 2 SMS-IP, 3 RCS, 4 PSVT. + +Status (`RegistrationReason`, its CP reason; ImsReasonInfo code, extra): + +| Status | Name | Code / extra | Stock reaction | +| --- | --- | --- | --- | +| 0 | REGISTER_SUCCESS | 132 | | +| 1 | DEREGISTER_FEATURE_MISMATCH | 132 | | +| 2 | DEREGISTER_NETWORK_TRIGGERED | 132 / 1513 | keep waiting | +| 3 | DEREGISTER_NO_RESP_FROM_NETWORK | 132 / 1503 | PDN torn down | +| 4 | DEREGISTER_INTERNAL | 132 / 131 | | +| **5** | **REGISTER_FAIL** | **1000 / 61442** | **PDN torn down (`isNeedPdnDeactivationReason`); not a CP retry reason** | +| 6 | REGISTER_FAIL_500 | 1000 / 352 | | +| 7 | REGISTER_FAIL_403 | 1000 / 332 | CP retries; PDN down unless 403 retry applies | +| 8 | REGISTER_FAIL_RETRY | 1000 / 61441 | CP retries, keep waiting | +| 9 | REGISTER_LIMITED | 132 / 121 | SMS-IP feature cleared | +| 10 | FAIL_TRY_AFTER_CELL_CHANGE | 1000 / 61442 | | +| 11 | FAIL_PLMN_BARR | 132 / 1000 | keep waiting | +| 12 | FAIL_FRESH_REG_START | 132 / 1000 | keep waiting | +| 13 | DEREGISTER_FAIL_ONGOING_CALL | 1000 | | +| 14 | FAIL_START_TIMER | | | +| 15 | FAIL_TRY_AFTER_WIFI_REASSOCIATE | 1000 | | +| 16 | FAIL_TRY_AFTER_RAT_CHANGE | 132 | | +| 18 | FAIL_SILENT_RETRY_FOR_CALL | 132 | | +| 254 | FAIL_NO_VALID_SIM | 1000 / 1501 | | +| 512 | RETRY_AFTER_PLMN_CHANGE | 132 / 1000 | | +| 255 | unspecified | | | + +Status 5 arriving immediately after ADD_PDN, with no SIP exchange in +between, fits a stack with no P-CSCF to send REGISTER to: the CP gives up +for that PDN at once. A SIP failure would come back as 6, 7 or 8 after +seconds, usually with reason text. + +## 8. Requests the AP owes the CP + +- **No AP-answered identity requests.** The CP never asks the service for + IMEI/IMEISV; the IMEISV only goes into the user agent if the template has + `#IMEISV#` (Verizon's doesn't). `RILC_REQ_IMS_SIM_AUTH` and + `GET_GBA_AUTH` are AP-to-CP requests for XCAP/GBA, not something the CP + waits for. +- **Time:** AIMS_SET_FRAME_TIME operation 2 with the wall clock, right after + stack start. We don't send it (0x0d0b isn't on `sit.py`'s allow-list). + Registration doesn't obviously depend on it (T-Mobile registered without + it), but it is the one start-up handshake we skip. +- **Carrier-config handshake:** the 0x4601 status before stack start; done + since 0.3.19. +- **Location/country:** only for Wi-Fi registration (ADD_PDN's Wi-Fi fields, + AIMS_SET_GEO_LOCATION_INFO for emergency); nothing on cellular. + +## 9. omamobile-modem-sit 0.3.20 (implemented, uncommitted, tested offline) + +- `data_call.setup_payload(..., ims=True, ip_type=...)`: the IMS bearer's PDP + type and P-CSCF request now follow `ip_type` (0xda = IP type, 0xdb = + `PCSCF_REQUEST`: IPv6 2, IPv4v6 3). `ip_type=None` keeps 0.3.19's IPv6/2. +- `bringup.ims_ip_type`: the IMS bearer uses the home IP type of the + `[profiles]` entry with the `ims` type for `ims_apn` (read only), else IPv6. + With the Verizon example profile (`2 = IMS; types=ia,ims; ip=ipv4v6; + roaming_ip=ipv6`) the IMS setup becomes PDP type 3 with P-CSCF request 3, + the same as the attach on that APN and as stock asks for Verizon's IMS APN + at home. T-Mobile (no such profile) is unchanged. +- `ims.add_pdn_payload`: IP type from the bearer's addresses (stock's + `getIpType`), so an IPv4v6 bearer is announced with its IPv4 address, + IPv4 P-CSCFs (at 39) and IPv4 DNS. An IPv6 bearer gives the same bytes as + 0.3.19 (tested). The empty-list path stays behind `ims_pcscf_required = no`, + now documented as something stock never does. +- `ims.oma_dm_menu` sent before ADD_PDN: `00 06 00 | 09 03 01 00 00 00` + (tag 777 = 1). Stock's empty AMR items 782/783 are left out on purpose. +- ISIM name resolution adds stock's path: the **internet bearer's own DNS + servers** (from its setup response, bound to its IPv6 address), after the + IMS bearer and before the host resolver + (`bringup.resolve_default_network`). The host resolver may be the USB + link to the laptop, which is not what stock would ask. +- IND_AIMS_REGISTRATION is logged with the state and status names. +- `packages/omamobile-modem-sit/test/all` passes (unit tests and the RIL + against `tools/sit-sim`). + +## 10. Ranked changes and runbook + +| # | Change | Why | Risk | +| --- | --- | --- | --- | +| 0 | **Read stock's answer first** (human's call: needs a stock boot of slot A, and then a fresh read-only NV backup per AGENTS.md). With this SIM on stock, `adb logcat -b all` filtered for `ShannonIms`/`SLogger`: "Sending AddPdn Info IpType [...]", "P-CSCF address of LinkProperties is invalid. Use P-CSCF address received from SIM", "P-CSCF Address:", "stop registration for invalid pcscf", "pcscf-ip-address-6-num". Tells whether Verizon's PCO carries the P-CSCF on stock and whether the SIM name resolves | removes guessing | stock boot: medium (rewrites `efs`/`persist`/`modem_userdata`) | +| 1 | Set `ims_pcscf_required = yes` on the phone (the package default) | an empty list gets status 5 by construction; stock never sends it | none | +| 2 | 0.3.20: IMS bearer IPv4v6 from the IMS profile (SETUP_DATA_CALL `0xda=3, 0xdb=3`), and ADD_PDN with IP type 3 | matches stock's request on Verizon's IPv4v6 IMS APN, which is also the attach APN (attach already asks 3/3); a mismatched IPv6-only request on the attach PDN's APN may be why the CP reports no PCO P-CSCF | medium: changes the IMS bearer request on Verizon only | +| 3 | 0.3.20: resolve the ISIM's FQDN through the internet bearer's DNS | stock's `getPcscfAddressFromSim` path | low (DNS queries only) | +| 4 | 0.3.20: OMA-DM menu (777 = 1) before ADD_PDN | stock always sends it; a Verizon-profile CP may expect it | low | +| 5 | Log what the CP sends after attach: P-CSCF family/counts of every entry of GET_DATA_CALL_LIST (not only CID 2), and IND_PCO_DATA (0x060d) container IDs and lengths only | shows whether the attach PDN's PCO has a P-CSCF the CID 2 response drops | none (read-only) | +| 6 | AIMS_SET_FRAME_TIME `00 02 ` after STACK_START (needs 0x0d0b on the allow-list: maintainer's call) | stock's start-up handshake | low | +| 7 | Verizon user agent `Google---Pixel 7 Pro---` in tag 519 | Verizon's template; matters only once SIP REGISTER goes out | low | +| 8 | Hidden-menu part 2 (video media) and XCAPM_START | completeness; not needed for registration | low | + +Runbook for 0.3.20 on the phone (human approves each step; no reboot needed +beyond restarting the service, which the phone's owner should know about): + +1. Read-only: confirm `[profiles]` has the IMS entry with `ip=ipv4v6` and + `ims_pcscf_required = yes` in `/etc/omamobile/modem-sit.conf`. + Pass: both present. Stop: no IMS profile (0.3.20 then behaves as 0.3.19). +2. Install 0.3.20 (`tools/phone-install-package` from this worktree), restart + `omamobile-sitd` after an IMS-free boot (the daemon refuses a second IMS + attempt in one boot). Expect in the journal: "IMS bearer PDP type ipv4v6", + "data-call status 0 ... PDP type 3", "P-CSCF count IPv4 n IPv6 m". + Pass: a count above 0, ADD_PDN sent, then IND_AIMS_REGISTRATION state 2. + Stop: setup status non-zero, or counts 0 and the ISIM name unresolved + (ADD_PDN is then not sent; no status 5). Rollback: reinstall 0.3.19. + Risk: medium (new IMS bearer request; internet bearer untouched). +3. If counts stay 0: change 5 (logging), then step 0. diff --git a/device/cheetah/MODEM-PROVIDER-PROFILES.md b/device/cheetah/MODEM-PROVIDER-PROFILES.md new file mode 100644 index 000000000..56d7dfc52 --- /dev/null +++ b/device/cheetah/MODEM-PROVIDER-PROFILES.md @@ -0,0 +1,238 @@ +# Provider profiles, APNs and the stock bring-up order (cheetah, S5300) + +Status (2026-11-02, offline research, nothing run): what stock Android does +for a SIM from modem boot to IMS registration, where its APN and carrier +data comes from, what our RIL (`omamobile-modem-sit`) does differently, and a +design for doing it the stock way on every carrier. Sources: the owner's +factory image (read on the laptop, never committed), libsitril and the SIT +stream library (function names and offsets), Shannon's IMS service APK +(class and key names), Google's CarrierSettings protobufs and the modem's +carrier-config database. Only field names and short values such as APN +names are quoted. Confidence is given per finding. + +## 1. Two separate carrier databases + +Stock keeps carrier data in two places, read by two different consumers. + +| | AP side: CarrierSettings | CP side: modem carrier config | +|---|---|---| +| Files | `product/etc/CarrierSettings/*.pb` (about 620 files) and `carrier_list.pb` | `vendor/firmware/carrierconfig/`: `cfg.db`, `manifests/`, `confseqs/` | +| Format | protobuf, one file per carrier (`_.pb`) | SQLite (matching) plus protobuf manifests and configuration sequences | +| Matched by | `carrier_list.pb`: MCC/MNC plus optional GID1, SPN, IMSI prefix | `cfg.db` `carrier_info`: MCC/MNC plus GID1, GID2, SPN, IMSI and ICCID patterns, ranked | +| Holds | APNs with types, protocols, bearers, profile IDs; Android carrier config keys (IMS settings, SIP domain, signal thresholds, 5G icons ...) | NV settings for the modem: hashed NV item IDs and values (protocol timers, CA/ENDC band combinations, SRS, PLMN mapping, carrier-specific items) | +| Consumer | Android telephony (DataProfileManager, ImsService via CarrierConfigManager) | the CP, through `OEM_SET_CARRIER_CONFIG_MANIFEST` (0x4600) and rfsd | + +**APNs are only on the AP side.** The modem's manifests and confseqs carry +no APN strings: they are NV item settings (`confseqs` decode as protobuf: +version, name such as `vzw.sim1`, then repeated `{hashed item ID, value}`). +Confidence high. + +### 1.1 This SIM's entries + +- **AP (CarrierSettings):** `carrier_list.pb` maps the SIM's MVNO GID1 on + the Verizon codes to `tracfoneverizon_us`. Its APN list is identical to + `verizon_us` (decoded, field by field): + + | APN | types | bearers | protocol (home/roaming) | profile ID | modem-cognitive | + |---|---|---|---|---|---| + | `internet` | default, mms, supl, dun, fota, cbs | CDMA/EVDO only | IPv4v6 | - | - | + | `VZWINTERNET` | default, supl, dun | GSM/UMTS/LTE/IWLAN/NR | IPv4v6 | 0 | yes | + | `VZWADMIN` | fota | as above | IPv4v6 | 3 | yes | + | `IMS` | **ia**, ims | LTE, IWLAN, NR | IPv4v6 / IPv6 | 2 | yes | + | `VZWAPP` | cbs, mms | as above | IPv4v6 | 4 | yes | + | `VZWEMERGENCY` | emergency | - | IPv4v6 / IPv6 | - | - | + + So stock's **initial attach APN is `IMS`** (the `ia` type), not `vzwims`, + and the internet APN is `VZWINTERNET`. The carrier config also sets the + IMS phone-context domain to the carrier's IMS domain and the usual + Verizon IMS keys. +- **CP (cfg.db):** the same GID1 ranks `Tracfone-VZW` (carrier ID 10008, + manifest `us_tracfone`) above plain Verizon (1839, `us_vzw`). The two + manifests share 32 configuration sequences. `us_tracfone` adds + `tracfone.sim1/sim2` (one NV item each) and drops `vzw-only.sim1/sim2/ + multislot` and `vzw-0x18F.sim1/sim2` (Verizon-only timers and items). + Neither contains APNs. + +## 2. The stock bring-up order + +From libsitril's start-up hooks (MODEM-SIT-RESEARCH section 4), the IMS +service's state names and AOSP telephony's behaviour. Order across services +is inferred, not captured (a stock radio log would settle it). + +1. **CP online** (cbd, rfsd). rild opens IPC0. +2. **Radio available** (`MiscService::OnRadioAvailable` and friends): + SET_DEBUG_TRACE off, SET_MODEM_CONFIG, SET_LOGICAL_TO_PHYSICAL_SLOT_MAPPING, + SET_SGC (`persist.vendor.radio.sgc`), the software version, SET_DEVICE_INFO, + SET_AP_SYSTEM_TIME; `PsService::OnRadioAvailable` re-applies the stored + initial attach APN; `NetworkService::OnRadioAvailable` sends + **SET_RADIO_POWER on** (unless airplane mode). The radio is turned on + once and stays on. +3. **SIM loads.** The CP sends `OEM_IND_CARRIER_CONFIG_SIM_INFO` (0x4602). + `MiscService::OnOemCpCarrierConfigSimInfoInd` (0x137c90) queues + `DoOemSetCpCarrierConfig` (0x137ed0): it matches the SIM in `cfg.db` and + sends 0x4600 (directory string `/vendor/firmware/carrierconfig`, the + manifest ID decoded from hex; built by + `ProtocolMiscBuilder::BuildSetCpCarrierConfig`). Stock never sends 0x4600 + unprompted. +4. **The CP applies the profile** and answers `OEM_IND_CARRIER_CONFIG_STATUS` + (0x4601, status and cause). `OnOemCpCarrierConfigStatusInd` (0x138e70) + logs it and forwards it to the IMS service (`RilIndCpCarrierConfigChanged`, + `MANIFEST_UPDATED` / `MANIFEST_NOT_UPDATED`). The selection is kept in + `persist.vendor.radio.config.carrier_id` on the AP and, on the CP, in its + NV. +5. **When a profile changes NV, the CP asks for a reset.** libsitril's + `PhoneResetHandler` handles `IND_PHONE_RESET` (0x0804) with reasons + including `GCONFIG`, `DEFAULT CONFIG`, `SGC CHANGE`, `UPDATE RSP`, `SIM`, + `RADIO POWER`, and sends a device reset request. On stock the reset is a + CP restart with the new NV in place; on the next start the manifest + matches what NV holds and no reset is needed. (Confidence medium: the + reasons and the handler are certain, the exact trigger after a manifest + change is inferred.) +6. **Android telephony, after the carrier config is loaded:** the initial + attach APN (`ia`: `IMS`) with SET_INITIAL_ATTACH_APN; **every + modem-cognitive data profile** (IDs 0, 2, 3, 4 above) with + SET_DATA_PROFILE; the preferred network type bitmap; data allowed + (SET_PS_SERVICE / mobile data). Setup data calls follow as Android needs + them: IMS first (by the IMS service), internet when mobile data is on. +7. **IMS:** the IMS service holds its stack start + (`mImsStackStartBlocked`, "IMS stack start has been blocked during + Carrier Config Manifest") until step 4 has reported, then + AIMS_STACK_START_REQ, the IMS PDN (on Verizon: the attach PDN, APN `IMS`), + AIMS_ADD_PDN_INFO with the P-CSCF list (from the PCO, with the ISIM's + EF_PCSCF added: `addPcscfFromSim`, `getIsimPcscf`), and the modem's IMS + stack registers. For this network the IMS service's own config + (`sim_configuration_311.xml`, MNC 480) does not block the PDN on a + missing P-CSCF (`BLOCK_PDN_BY_INVALID_PCSCF` 0) and requires a single + registration. + +## 3. What ours does differently (0.3.14) + +| Step | Stock | omamobile-modem-sit | +|---|---|---| +| Radio | on once, stays on | AT+CFUN=0, attach APN, data profile, AT+CFUN=1: a radio cycle at every start | +| Carrier profile | 0x4600 on 0x4602, then a CP reset if NV changed | none by default; `carrier_manifest` sends it, but nothing handles the reset request, and a reset can't be done in place (it needs a reboot, which discards the NV RAM copy and with it the profile) | +| Attach APN | the `ia` APN from CarrierSettings (`IMS`) | `attach_apn` by hand (`vzwims` tonight) | +| Data profiles | every modem-cognitive profile with its profile ID | one default profile (ID 0) | +| NR | per carrier config | SET_NR_MODE 3 -> 0 **after** registration, which can make the CP re-register | +| IMS stack start | after the carrier-config status | immediately (0.3.14: waits only when `carrier_manifest` is set) | +| P-CSCF | PCO, plus the ISIM | PCO only; empty on this network | +| NV | persistent | a RAM copy of the backup, dropped at stop: every boot starts from the same old state | + +0.3.15 (built offline, not yet run on a phone) takes three rows to stock's +way: it reads the radio state (AT+CFUN?) and sends the attach APN, every +profile in the new `[profiles]` section (profile 0 from `apn` unless +`[profiles]` has its own) and NR mode 0 before radio on, switching the radio +on once and never off; it logs IND_PHONE_RESET's first payload bytes raw and +the time from radio on to the first packet registration. + +## 4. Why registration takes minutes + +In order of likely weight: + +1. **The NV RAM copy is the backup's state at every boot.** The CP keeps its + acquisition data (recent PLMNs, bands, cells) and the carrier profile in + NV. Ours starts each boot from the same backup, taken without this SIM, + so the CP does a cold search every time, with whatever carrier profile + the backup holds (not Verizon's). Stock's first boot with a new SIM is + slow too; every later boot starts from the last cell. Confidence medium + (the acquisition database's location is not read; the behaviour fits + 34 s to 10+ min with the same cell). +2. **The radio cycle and the late NR change.** AT+CFUN=0/1 at start and + SET_NR_MODE after registration both make the CP search or re-register; + stock does neither at start. +3. **A profile mismatch.** Without Verizon's NV settings the CP runs a + generic profile; after the Tracfone manifest the CP applied it (status 1) + and stopped searching (CEREG 0,0), consistent with it waiting for the + reset of step 2.5 that sitd never answered (0x0804 is not logged yet). +4. **IND_REG_FAILED cause 10 after the data permission** (seen on both + phones): the network detaches the phone once around the first data + setup; the settle wait of 0.3.8 rides it out, but each costs a re-attach. + +## 5. IMS on this network + +- **The PDN:** IMS runs on the attach PDN (APN `IMS`, types `ia` and `ims`). + A second IMS setup on the same APN returns that PDN. +- **The P-CSCF:** Verizon delivers it in the PCO of the IMS PDN when the UE + asks for it; whether the CP asks, and in which PCO containers, is CP + configuration (the carrier profile). With the generic profile the PCO + comes back without it, even with the P-CSCF request bit set in our attach + (0.3.11). The ISIM's EF_PCSCF is stock's second source. +- **What is missing, most likely:** the carrier profile in effect on the CP + (step 2.3-2.5). Without it the CP's IMS stack has no carrier settings, and + ADD_PDN with an empty P-CSCF list was accepted but never led to a + registration (0.3.13). + +## 6. Design for Omarchy Mobile (every carrier) + +**Data sources (all from the owner's own factory image, extracted on the +laptop like the vendor chroot, never hosted or committed):** + +- AP: CarrierSettings protobufs, read with a small decoder for the fields we + use (APN name, types, protocols, bearer bitmask, profile ID, + modem-cognitive; a handful of carrier config keys). This is the database + stock uses; it has the `ia` APN for each carrier. +- CP: `vendor/firmware/carrierconfig` in the modem chroot (0.3.14's + extractor already copies it). +- Fallback where the image has no entry: mobile-broadband-provider-info + (no `ia` types, so its attach APN is a guess), then a small override file + in the package, then the user's own setting. + +**Matching, with minimal identity handling:** MCC/MNC, GID1 and SPN are what +both databases match on. Read them once, in sitd, keep only those fields in +memory, log only the matched carrier names. Options: GET_IMSI's first 5-6 +digits plus GID1 through SIM I/O, or the CP's 0x4602 (which carries them, but +also full identities). This needs the maintainer's decision; until then the +carrier can be chosen by hand (the 0.3.14 `carrier_manifest` setting, and the +same for the CarrierSettings name). + +**Order (stock's):** + +1. CP online. Radio stays off. +2. Wait for 0x4602; answer with the CP manifest; wait for 0x4601. +3. If the CP asks for a reset (0x0804): this needs NV persistence (below). +4. SET_INITIAL_ATTACH_APN with the `ia` APN; SET_DATA_PROFILE with every + modem-cognitive profile and its profile ID; preferred network type and NR + mode **before** radio on. +5. Radio on, once. Wait for registration. +6. Data permission, settle, internet bearer. +7. AIMS stack start, IMS bearer (the attach PDN where `ia` = `ims`), ADD_PDN + with PCO P-CSCFs, else the ISIM's. + +**NV persistence: the decision this hinges on.** Stock's carrier profile and +acquisition data live in NV. Our profile works on a RAM copy that is dropped +at stop, which keeps the partitions safe but makes every boot a first boot +with the backup's carrier. Options, for the maintainer: + +- (a) Keep RAM-only and seed NV from a stock boot with the SIM in: boot + stock once with this SIM, let it register and apply the profile, take a + fresh read-only backup (AGENTS.md section 1), update `modem-nv.sha256`. + The RAM copy then starts every boot with Verizon's profile and a recent + cell. No new write path; repeat per SIM change. +- (b) A persistent copy of the modem's NV kept off the partitions (for + example under /var/lib), checksum-verified like the RAM copy, so the CP's + changes survive reboots. The partitions are still never written; it + changes the "loses what the modem writes" rule. +- (c) Never apply a profile and live with the generic one: no IMS on + networks that need their profile. + +**Where PR #373 fits:** its carrier lookup (MCC/MNC from GET_IMSI into +mobile-broadband-provider-info plus overrides) is the fallback layer above, +and its Mobile network page is the place to show and override the matched +carrier. Changes it needs: read CarrierSettings first (it has the `ia` APN: +`IMS` for Verizon, where the override guesses `vzwims`); send every +modem-cognitive profile, not only the default; match on GID1 too (MVNOs on +shared codes, as here); and order the requests as above. + +## 7. Next steps, by expected gain + +1. **Seed NV from a stock boot with the SIM** (option a) and boot ours + without a manifest or radio cycle changes: tests whether registration + gets fast and whether IMS registers once the CP carries the profile. + Risk: medium (a stock boot rewrites efs, persist and modem_userdata; the + backup and checksum steps of AGENTS.md section 1 are mandatory). +2. **Stock's order and APNs:** attach on `IMS` (not `vzwims`), all + modem-cognitive profiles, NR mode set before radio on, no AT+CFUN cycle + when the radio is already off at start. Risk: low. +3. **Handle and log 0x0804 (IND_PHONE_RESET)** with its type and reason, so + the next manifest test shows whether the CP asked for a reset. Risk: + none (logging only). diff --git a/device/cheetah/MODEM-SIT-RESEARCH.md b/device/cheetah/MODEM-SIT-RESEARCH.md index a4427e09e..637e41016 100644 --- a/device/cheetah/MODEM-SIT-RESEARCH.md +++ b/device/cheetah/MODEM-SIT-RESEARCH.md @@ -207,7 +207,7 @@ START_NETWORK, 1 for STOP_NETWORK. | 0x0202 / 0x0204 | VERIFY_SIM_PUK / PUK2 | not read; same family as above | low | names | | 0x0210 | IND_SIM_STATUS_CHANGED | re-read GET_SIM_STATUS | medium | name | | 0x0400 | GET_IMSI | request len 29: 0x0c u8 AID length, 0x0d AID (16 bytes). Response: 0x0c u8 IMSI length (at most 15), 0x0d IMSI as ASCII digits | high | `ProtocolSimBuilder::BuildGetImsi`, `ProtocolSimImsiAdapter::Init` | -| 0x0208 | SIM_IO | ICCID is read the Android way, through SIM_IO on EF_ICCID (2FE2); layout not read yet | low | name; `SimIoService` in libsitril | +| 0x0208 | SIM_IO | request len 572: 0x0c u8 command (B0/B2/C0/D6/DC/F2), 0x0d u8 application type (RIL_AppType, from the AID's entry in the SIM status; ISIM 5), 0x0e u16 file ID (LE), 0x10 u8 path length in bytes, 0x11 path (12 bytes, each 16-bit ID little-endian: 3F007FFF is 00 3F FF 7F), 0x1d P1, 0x1e P2, 0x1f P3, 0x20 u16 data length, 0x22 data (512), 0x222 u8 PIN2 length, 0x223 PIN2 (8), 0x22b u8 AID length, 0x22c AID (16). Response: 0x0c SW1, 0x0d SW2, 0x0e s16 response length (at most 512), 0x10 response. omamobile-modem-sit 0.3.18 sends only GET RESPONSE and READ RECORD on the ISIM's EF_PCSCF (6F09), checked in `sit.check` | high (offsets), medium (application type values, the GET RESPONSE format the CP returns) | `ProtocolSimBuilder::BuildSimIO`, `ProtocolSimIOAdapter::GetSw1/GetSw2/GetResponseLength/GetResponse`, `SimIoService::DoSimIo` | | 0x0401 / 0x0402 | GET_IMEI / GET_IMEISV | bare header (len 12) | high | `ProtocolDeviceInfoBuilder::GetIMEI/GetIMEISV` | | 0x0250 | SET_LOGICAL_TO_PHYSICAL_SLOT_MAPPING | len 17: 0x0c u8 count, 0x0d.. one u8 physical slot per logical slot (4 bytes) | high (layout), medium (meaning) | `ProtocolRadioConfigBuilder::BuildSimSetLogicalSlotMapping` | diff --git a/packages/omamobile-mmd/lib/bridge.py b/packages/omamobile-mmd/lib/bridge.py index 5dcbab777..462c7e52b 100644 --- a/packages/omamobile-mmd/lib/bridge.py +++ b/packages/omamobile-mmd/lib/bridge.py @@ -92,6 +92,8 @@ def derive(status): plmn=m.get('plmn', '') if registered else '', operator_name=m.get('operator_name', '') if registered else '', home=reg == 'home', + # sitd's LTE RSRP as 0-100 while registered; "recent" while it has one. + signal=max(0, min(100, int(m.get('signal') or 0))) if registered else 0, emergency_numbers=list(m.get('emergency_numbers') or ('112', '911')), ) return d @@ -215,7 +217,7 @@ def static_props(self, name): CurrentCapabilities=mm.CAP_GSM_UMTS | mm.CAP_LTE, MaxBearers=1, MaxActiveBearers=1, SupportedModes=[(mm.MODE_4G, mm.MODE_NONE)], CurrentModes=(mm.MODE_4G, mm.MODE_NONE), SupportedBands=[mm.BAND_UNKNOWN], CurrentBands=[mm.BAND_UNKNOWN], - SupportedIpFamilies=mm.IP_V4 | mm.IP_V6 | mm.IP_V4V6, SignalQuality=(0, False)) + SupportedIpFamilies=mm.IP_V4 | mm.IP_V6 | mm.IP_V4V6) if name == mm.MODEM_3GPP: apn, ip_type = self.settings.get('apn', ''), self.settings.get('ip_type', 'ipv4v6') settings = {'apn': GLib.Variant('s', apn), @@ -232,7 +234,7 @@ def dynamic_props(self, name, d): return dict(Sim=self.sim or '/', Bearers=[self.bearer] if self.bearer else [], State=d['state'], StateFailedReason=d['failed'], UnlockRequired=d['unlock'], UnlockRetries=d['retries'], PowerState=d['power'], - AccessTechnologies=d['access']) + AccessTechnologies=d['access'], SignalQuality=(d['signal'], d['signal'] > 0)) if name == mm.MODEM_3GPP: return dict(RegistrationState=d['registration'], OperatorCode=d['plmn'], OperatorName=d['operator_name'], PacketServiceState=d['packet']) diff --git a/packages/omamobile-mmd/test/test_bridge.py b/packages/omamobile-mmd/test/test_bridge.py index bd44a34d1..e3bcedaf4 100644 --- a/packages/omamobile-mmd/test/test_bridge.py +++ b/packages/omamobile-mmd/test/test_bridge.py @@ -40,6 +40,12 @@ def test_states(self): d = bridge.derive(status) self.assertEqual((d['state'], d['failed']), (state, failed), status['modem']['sim']) + def test_signal_only_while_registered_and_clamped(self): + self.assertEqual(bridge.derive(registered_status(signal=32))['signal'], 32) + self.assertEqual(bridge.derive(registered_status(signal=250))['signal'], 100) + self.assertEqual(bridge.derive(registered_status(signal=None))['signal'], 0) + self.assertEqual(bridge.derive(blank_status(sim='ready', radio='on', signal=40))['signal'], 0) + def test_details(self): d = bridge.derive(blank_status(sim='pin')) self.assertEqual((d['unlock'], d['emergency_only'], d['sim_object'], d['interfaces']), diff --git a/packages/omamobile-mmd/version b/packages/omamobile-mmd/version index 6e8bf73aa..17e51c385 100644 --- a/packages/omamobile-mmd/version +++ b/packages/omamobile-mmd/version @@ -1 +1 @@ -0.1.0 +0.1.1 diff --git a/packages/omamobile-modem-sit/bin/omamobile-sit b/packages/omamobile-modem-sit/bin/omamobile-sit index 4298f5e14..9b86a4185 100755 --- a/packages/omamobile-modem-sit/bin/omamobile-sit +++ b/packages/omamobile-modem-sit/bin/omamobile-sit @@ -12,7 +12,7 @@ omamobile-sit sms send NUMBER TEXT one SMS (several parts if needed) omamobile-sit sms list [--json] received, not yet deleted omamobile-sit sms delete ID - omamobile-sit data on|off the cellular fallback route (rmnet0) + omamobile-sit data on|off the cellular fallback route (rmnet) omamobile-sit radio on|off AT+CFUN 1/0 (registration and bearers go with "off"; a restart needs a reboot) diff --git a/packages/omamobile-modem-sit/bin/omamobile-sit-data b/packages/omamobile-modem-sit/bin/omamobile-sit-data index e256494af..a42aac1d2 100755 --- a/packages/omamobile-modem-sit/bin/omamobile-sit-data +++ b/packages/omamobile-modem-sit/bin/omamobile-sit-data @@ -1,6 +1,6 @@ #!/usr/bin/env python3 # SPDX-License-Identifier: MIT -# omamobile-sit-data: the cellular fallback route on rmnet0 (lib/data_network.py). +# omamobile-sit-data: the cellular fallback route on the bearer's rmnet device (lib/data_network.py). import os import sys diff --git a/packages/omamobile-modem-sit/etc/omamobile/modem-sit.conf b/packages/omamobile-modem-sit/etc/omamobile/modem-sit.conf index a55b4fc90..b4996d613 100644 --- a/packages/omamobile-modem-sit/etc/omamobile/modem-sit.conf +++ b/packages/omamobile-modem-sit/etc/omamobile/modem-sit.conf @@ -6,6 +6,13 @@ # The carrier's internet APN, as the carrier publishes it. Required: without # it the start-up stops before the radio is switched on. apn = +# The APN of the initial LTE attach, if the carrier attaches on another APN +# than its internet APN (some attach only on their IMS APN, e.g. "ims"; the +# internet bearer still uses apn). Empty: the same as apn. +# The attach APN, the data profiles ([profiles] below) and the NR mode are +# sent before the radio is switched on, or with it on if it already is: the +# radio is never cycled. +attach_apn = # The internet bearer's IP type: ipv4v6, ipv4 or ipv6. The cellular route # (omamobile-sit-data) is IPv4 only for now. ip_type = ipv4v6 @@ -17,14 +24,54 @@ ims_apn = ims # without SMS or calls. ims_required = yes ends start-up instead. ims_required = no ims_timeout = 120 +# The IMS registration needs a P-CSCF from the IMS bearer. "no" registers +# without one when the bearer carries none, leaving it to the modem's IMS +# stack (some networks keep it on the ISIM, not in the bearer). +ims_pcscf_required = yes +# When the IMS bearer carries no P-CSCF, read it from the ISIM's EF_PCSCF +# (one read-only file; nothing else on the ISIM is read), as stock does. +# A name there is resolved to IPv6 over the IMS bearer, else through the +# host's resolver. +ims_pcscf_from_isim = yes +# The modem's carrier profile: the 40-character cfg.db manifest ID for this +# SIM's carrier (a file name under the modem chroot's +# /vendor/firmware/carrierconfig/manifests). Set, it answers the modem's +# request for a carrier config once, and the IMS stack start waits up to +# carrier_manifest_timeout seconds for the modem's answer. Empty: not sent. +carrier_manifest = +carrier_manifest_timeout = 60 # nr = off keeps the modem on LTE (NR mode byte 3 -> 0; the original is kept -# in RAM). On this phone NR registered but carried no traffic. "on" leaves -# the modem's own setting alone. +# in RAM), set before the radio is switched on. On this phone NR registered +# but carried no traffic. "on" leaves the modem's own setting alone. nr = off # Seconds to wait for the SIM, then for packet registration. sim_timeout = 120 register_timeout = 180 +[profiles] +# Extra data profiles the modem manages itself (SET_DATA_PROFILE), sent with +# the attach APN before the radio is switched on, as stock sends its +# carrier's "modem-cognitive" APNs. Empty (the default): only profile 0, the +# internet APN above. One line per profile: +# +# ID = APN; types=TYPE,TYPE; ip=IP; roaming_ip=IP +# +# ID: 0 default/internet (replaces the one made from apn; the internet +# bearer still uses apn), 2 IMS, 3 FOTA, 4 CBS (1 tethered, 1000 and up +# OEM). +# types (required): default, mms, supl, dun, hipri, fota, ims, cbs, ia, +# emergency, mcx, xcap. +# ip: ipv4v6 (default), ipv4 or ipv6; roaming_ip: the same choices, default ip. +# No authentication. Each profile gets stock's limits: 20 connections in +# 300 s, no wait, enabled, any bearer. +# +# Example, a carrier that attaches on its IMS APN (with attach_apn = IMS, +# ims_apn = IMS and apn = VZWINTERNET above): +# 0 = VZWINTERNET; types=default,supl,dun; ip=ipv4v6 +# 2 = IMS; types=ia,ims; ip=ipv4v6; roaming_ip=ipv6 +# 3 = VZWADMIN; types=fota +# 4 = VZWAPP; types=cbs,mms + [data] # Reserved: the route's metric is fixed at 700 (Wi-Fi's default is 600). metric = 700 diff --git a/packages/omamobile-modem-sit/lib/at.py b/packages/omamobile-modem-sit/lib/at.py index 721453203..c51c9e132 100644 --- a/packages/omamobile-modem-sit/lib/at.py +++ b/packages/omamobile-modem-sit/lib/at.py @@ -1,9 +1,9 @@ # SPDX-License-Identifier: MIT """Fixed AT commands on CPIF's router channel (/dev/umts_router). -Only radio power goes this way: the start-up sequence that registered on -this phone switched the radio with AT+CFUN, and SIT radio power and AT CFUN -were seen to disagree. No arbitrary command can be sent from here. +Only radio power goes this way (and its read-only query): the start-up +sequence that registered on this phone switched the radio with AT+CFUN, and +SIT radio power and AT CFUN were seen to disagree. No arbitrary command can be sent from here. """ import os import re @@ -47,3 +47,17 @@ def radio(enabled): raise RuntimeError('Radio command was not accepted') finally: os.close(fd) + + +def radio_state(): + """AT+CFUN?: the functionality level (0 minimum, 1 full, 4 flight), or + None when the answer can't be read. Read-only.""" + fd = open_channel(ROUTER) + try: + response = exchange(fd, COMMANDS['radio-state'], 10) + finally: + os.close(fd) + found = re.search(r'\+CFUN:\s*(\d+)', response) + if not found or not re.search(r'(?:^|[\r\n])OK(?:[\r\n]|$)', response): + return None + return int(found.group(1)) diff --git a/packages/omamobile-modem-sit/lib/bringup.py b/packages/omamobile-modem-sit/lib/bringup.py index 45e2a98aa..3368d09ef 100644 --- a/packages/omamobile-modem-sit/lib/bringup.py +++ b/packages/omamobile-modem-sit/lib/bringup.py @@ -4,13 +4,24 @@ 1. wait for the USIM application to be READY (type 2, state 5); a PIN, PUK or network lock ends start-up at once (no PIN entry here); - 2. radio off (AT+CFUN=0), initial attach APN, the default data profile; - 3. radio on (AT+CFUN=1), wait for packet registration; - 4. NR mode 0 ([network] nr = off): stay on LTE (NR registered, but - carried no traffic on this phone), wait for LTE registration; - 5. the internet bearer (CID 1, rmnet0); - 6. IMS: stack start, the IMS bearer (CID 2, IPv6), ADD_PDN_INFO, then wait - for an IMS registration with voice and SMS. + 2. read the radio state (AT+CFUN?); then, with the radio as found, the + initial attach APN ([network] attach_apn, else apn), every data + profile (profile 0 from apn unless [profiles] has its own, then the + rest of [profiles]) and NR mode 0 ([network] nr = off: stay on LTE; NR + registered but carried no traffic on this phone); + 3. radio on (AT+CFUN=1) once, unless it was already on: stock's order, + with no radio cycle (device/cheetah/MODEM-PROVIDER-PROFILES.md + section 2). An unreadable radio state falls back to 0.3.14's cycle + (AT+CFUN=0 first). Then wait for packet registration (on LTE with + nr = off) and log how long it took; + 4. the internet bearer (CID 1; its rmnet device is found by omamobile-sit-data); + 5. IMS: stack start, the IMS bearer (CID 2; IPv6, or the IMS profile's + home IP type, as stock takes it from the IMS APN), the hidden menu and + the OMA-DM menu, ADD_PDN_INFO (its P-CSCF from the bearer's PCO, else + from the ISIM's EF_PCSCF with [network] ims_pcscf_from_isim = yes, a + name resolved over the IMS bearer, the internet bearer's DNS or the + host resolver), then wait for an IMS registration with voice and SMS. + The stock IMS service and its requests: device/cheetah/IMS-SHANNON.md. No retransmission of SMS/calls and no warm CP restart. A failed or uncertain step up to the internet bearer ends start-up; the service exits and the @@ -19,10 +30,43 @@ """ import time -from at import radio -from data_call import activate, initial_attach, provision_default_profile +from at import radio, radio_state +from data_call import APN_TYPES, activate, configured_profiles, initial_attach, provision_profiles +from data_call import paths as bearer_paths from ims import register, start_stack -from radio_settings import nr_mode +from isim import bearer_dns, ipv6_pcscf, resolve_on_bearer, resolve_system +from radio_settings import SAVED_NR, nr_mode +from sit import ModemError + + +def ims_ip_type(profiles, ims_apn): + """The IMS bearer's PDP type: the home protocol of the [profiles] entry + with the ims type for ims_apn, as stock takes it from the IMS APN + (Verizon's IMS APN is IPv4v6 at home), else IPv6 (unchanged from 0.3.19).""" + for profile in profiles: + if profile.types & APN_TYPES['ims'] and profile.apn.lower() == ims_apn.lower(): + return profile.ip + return 'ipv6' + + +def resolve_default_network(name): + """AAAA as stock's IMS service resolves an ISIM P-CSCF name + (InetAddress.getByName on the default network: the internet bearer + without Wi-Fi): the internet bearer's own DNS servers from its setup + response, then the host resolver, which may be another link.""" + found = [] + response = bearer_paths(1)[1] + if response.exists(): + bearer = response.read_bytes() + if len(bearer) >= 132 and any(bearer[9:25]): + source, servers = bearer_dns(bearer) + try: + found = resolve_on_bearer(name, source, servers) if servers else [] + except (OSError, ValueError): + found = [] + if found: + print(f'P-CSCF from ISIM: {name} resolved through the internet bearer\'s DNS', flush=True) + return found or resolve_system(name) def sim_ready(payload): @@ -60,9 +104,8 @@ def sim_lock(payload): def data_snapshot(client, label): - """Registration and the data-call list, as the modem reports them, before - and after the NR change: whether a bearer is ever listed before the - setup, and whether the change makes one appear or disappear. States, + """Registration and the data-call list, as the modem reports them at + registration: whether a bearer is ever listed before the setup. States, codes and counts only.""" try: reg = client.request(0x0701) @@ -112,6 +155,16 @@ def start(service, settings, stopped=lambda: False): apn, ip_type = net.get('apn', ''), net.get('ip_type', 'ipv4v6') if not apn: raise RuntimeError('No APN configured: set [network] apn in /etc/omamobile/modem-sit.conf') + # Some networks attach only on their IMS APN (2026-11-02, cheetah: with + # the internet APN as the attach APN the modem searched for 13 minutes + # with no reject; with the IMS APN it registered on LTE in 34 s). The + # data profile and the internet bearer keep apn. Empty: the same as apn. + attach_apn = net.get('attach_apn', '').strip() or apn + try: + profiles = configured_profiles(settings) + except ValueError as error: + raise RuntimeError(f'Bad data profile in /etc/omamobile/modem-sit.conf: {error}') from None + began = time.monotonic() previous = None locked = None @@ -145,20 +198,77 @@ def usim_ready(): 2: 'card error'}.get(previous[0] if previous else None, 'unknown card state') raise TimeoutError(f'No ready USIM within {timeout} s ({card})') from None service.stage('attach') - radio(False) - initial_attach(client, apn, ip_type) - provision_default_profile(client, apn, ip_type) + level = radio_state() + if level is None: + # Unknown: 0.3.14's known-good cycle rather than a guess. + print('Startup: radio state unreadable; switching it off before the attach settings', flush=True) + radio(False) + already_on = False + else: + already_on = level == 1 + print(f'Startup: radio state {level} ({"on: no radio cycle" if already_on else "off"})', flush=True) + # Attaching on the IMS APN: ask for the P-CSCF in the attach, as stock does + # for an IMS APN, since that bearer is the one IMS then uses. + ims_attach = (attach_apn != apn and net.getboolean('ims', True) + and attach_apn == net.get('ims_apn', 'ims')) + if attach_apn != apn: + print('initial attach on attach_apn, not the internet APN' + + (' (the IMS APN: P-CSCF requested)' if ims_attach else ''), flush=True) + initial_attach(client, attach_apn, ip_type, ims=ims_attach) + if profiles[0].ident == 0 and profiles[0].apn != apn: + print('data profile 0 from [profiles], with another APN than [network] apn ' + '(the internet bearer uses apn)', flush=True) + provision_profiles(client, profiles) + lte = net.get('nr', 'off') == 'off' + nr_after_radio = False + if lte: + # Before radio on, as stock sets its network type: changed after + # registration it made the CP register again. This modem refuses + # SET_NR_MODE with the radio off (error 2, 2026-11-02), so then it is + # set right after radio on, still before registration. + try: + nr_mode(client) + except ModemError as error: + if already_on: + raise + SAVED_NR.unlink(missing_ok=True) + nr_after_radio = True + print(f'NR mode refused with the radio off ({error}); setting it after radio on', flush=True) service.pump() - radio(True) + if not already_on and net.get('carrier_manifest', '').strip(): + # Stock applies the carrier profile before the radio comes on; with + # the radio switched on first (2026-11-02) the modem confirmed the + # profile 11 s later and then stopped searching. Wait for its status + # here, so registration starts with the profile in place. + limit = net.getint('carrier_manifest_timeout', 60) + try: + wait_for(service, lambda: service.carrier_status is not None, limit, stopped) + print('Radio on after carrier config: status', service.carrier_status[0], + 'cause', service.carrier_status[1], flush=True) + except TimeoutError: + print('Radio on without carrier config status after', limit, 's;', + 'manifest sent' if service.manifest_sent else 'no request from the modem seen', flush=True) + if not already_on: + radio(True) + began = time.monotonic() + if nr_after_radio: + nr_mode(client) service.stage('registering') - print('Startup: waiting for packet registration', flush=True) + print('Startup: waiting for packet registration' + (' on LTE' if lte else ''), flush=True) timeout = net.getint('register_timeout', 180) - wait_for(service, lambda: packet_registered(client.request(0x0701)), timeout, stopped) - data_snapshot(client, 'before the NR change') - if net.get('nr', 'off') == 'off': - nr_mode(client) - wait_for(service, lambda: packet_registered(client.request(0x0701), lte=True), 120, stopped) - data_snapshot(client, 'after the NR change') + first = None + + def registered(): + nonlocal first + state = client.request(0x0701) + if first is None and packet_registered(state): + first = time.monotonic() + print(f'Registration: packet registered {first - began:.1f} s after ' + + ('start-up began (the radio was already on)' if already_on else 'radio on') + + f', RAT {state[3]}', flush=True) + return packet_registered(state, lte=lte) + wait_for(service, registered, timeout, stopped) + data_snapshot(client, 'at registration') service.stage('data') activate(client, apn, ip_type=ip_type, on_event=service.event) service.pump() @@ -169,9 +279,32 @@ def usim_ready(): service.ims_state = 'pending' service.stage('ims') try: + if net.get('carrier_manifest', '').strip(): + # As stock's IMS service: no stack start while the modem applies + # its carrier profile. The modem's 0x4602 arrives on its own after + # the SIM loads; service.event answers it and records the 0x4601. + limit = net.getint('carrier_manifest_timeout', 60) + try: + wait_for(service, lambda: service.carrier_status is not None, limit, stopped) + print('IMS stack start released (after carrier config: status', + service.carrier_status[0], 'cause', service.carrier_status[1], ')', flush=True) + except TimeoutError: + print('IMS stack start released (timeout: no carrier config status in', limit, 's;', + 'manifest sent' if service.manifest_sent else 'no request from the modem seen', ')', + flush=True) start_stack(client) - activate(client, net.get('ims_apn', 'ims'), ims=True, on_event=service.event) - register(client) + ims_apn = net.get('ims_apn', 'ims') + ims_ip = ims_ip_type(profiles, ims_apn) + print('IMS bearer PDP type', ims_ip, '(from the IMS profile)' if ims_ip != 'ipv6' else '', flush=True) + activate(client, ims_apn, ims=True, ip_type=ims_ip, on_event=service.event) + fallback = None + if net.getboolean('ims_pcscf_from_isim', True): + # The ISIM's EF_PCSCF when the PCO has none (one read-only file). + def fallback(bearer): + return ipv6_pcscf(client, client.request(0x0200), bearer, + resolve_host=resolve_default_network) + register(client, pcscf_required=net.getboolean('ims_pcscf_required', True), + pcscf_fallback=fallback) print('Startup: waiting for IMS voice and SMS registration', flush=True) wait_for(service, lambda: service.registration().get('state') == 2 and service.registration().get('features', 0) & 3 == 3, diff --git a/packages/omamobile-modem-sit/lib/carrier_manifest.py b/packages/omamobile-modem-sit/lib/carrier_manifest.py new file mode 100644 index 000000000..ed112119e --- /dev/null +++ b/packages/omamobile-modem-sit/lib/carrier_manifest.py @@ -0,0 +1,65 @@ +# SPDX-License-Identifier: MIT +"""The modem's carrier profile, named by hand ([network] carrier_manifest). + +After the SIM loads, the CP sends OEM_IND_CARRIER_CONFIG_SIM_INFO (0x4602) +and waits for the AP to name a carrier manifest. Stock's libsitril answers +it with OEM_SET_CARRIER_CONFIG_MANIFEST (0x4600), only ever as that answer +(OnOemCpCarrierConfigSimInfoInd queues DoOemSetCpCarrierConfig with the +indication); the CP then reads the manifest and its configuration sequences +from the carrier-config directory through rfsd and reports the outcome in +OEM_IND_CARRIER_CONFIG_STATUS (0x4601). Stock's IMS service holds the AIMS +stack start until that status has come. + +Here the manifest ID is set by hand in the configuration, and 0x4602 is +only a trigger: its payload is never read. Empty setting: nothing is sent, +and 0x4600 stays on the transport's do-not-send list. + +0x4600 (stock ProtocolMiscBuilder::BuildSetCpCarrierConfig, 532 bytes with +the header): 0x0c u32 length and 0x10 the directory string (stock passes +CarrierConfigManager's "/vendor/firmware/carrierconfig"), 0x110 u32 length +and 0x114 the encoded manifest (CarrierManifest::getEncodedManifest: the +manifest's hex ID decoded to bytes). 0x4601 (stock +ProtocolMiscCarrierConfigStatusIndAdapter): 0x08 u32 status, 0x0c u32 cause. +""" +import os +from pathlib import Path +import re +import struct + +DIRECTORY = '/vendor/firmware/carrierconfig' +# The modem chroot's image on the host (omamobile-vendor-chroot's IMAGE). +HOST_DIRECTORY = Path(os.environ.get('OMAMOBILE_SIT_CARRIERCONFIG', + '/usr/lib/omamobile-vendor/modem/root' + DIRECTORY)) + + +def manifest_id(value, directory=None): + """The 20-byte manifest ID for a configured value, or raise ValueError + with the reason: exactly 40 hex characters, and manifests/ present + in the chroot's carrier-config directory (where the CP will read it).""" + value = (value or '').strip().lower() + if not re.fullmatch(r'[0-9a-f]{40}', value): + raise ValueError('carrier_manifest is not a 40-character hex manifest ID') + if not ((directory or HOST_DIRECTORY) / 'manifests' / value).is_file(): + raise ValueError('carrier_manifest names no manifest in the modem chroot ' + '(re-run tools/extract-vendor-chroot -p modem)') + return bytes.fromhex(value) + + +def request_payload(ident): + """The 0x4600 payload (the 532-byte request without its 12-byte header).""" + if len(ident) > 256: + raise ValueError('manifest ID too long') + payload = bytearray(0x214 - 12) + path = DIRECTORY.encode('ascii') + struct.pack_into('; data_network.find_device checks which device +really carries the internet bearer (it differed when the attach bearer was on +another APN). The APN comes from /etc/omamobile/modem-sit.conf. Ownership is recorded in RAM before a setup is sent: a setup or release with an unknown outcome @@ -37,6 +40,28 @@ def paths(cid): raise ValueError('unsupported bearer CID') +def pcscf_addresses(response): + """The P-CSCF addresses of a setup response or data-call item, as stock's + ProtocolPsSetupDataCallAdapter reads them: the base item's family byte at + 0x42 (1 IPv4, 2 IPv6, 3 both), three IPv4 at 0x43 and three IPv6 at 0x4f; + then, when the item is longer than the 132-byte base and its extension + version byte at 0x84 is 2 or more, five IPv4 at 0x86 and five IPv6 at + 0x9a (the 234-byte item). Empty slots are skipped, duplicates dropped. + Returns (IPv4 list, IPv6 list) of raw addresses; never log the values.""" + v4, v6 = [], [] + if len(response) >= 132: + family = response[0x42] + if family in (1, 3): + v4 += [bytes(response[i:i + 4]) for i in (0x43, 0x47, 0x4b)] + if family in (2, 3): + v6 += [bytes(response[i:i + 16]) for i in (0x4f, 0x5f, 0x6f)] + if len(response) >= 0xea and response[0x84] >= 2: + v4 += [bytes(response[i:i + 4]) for i in range(0x86, 0x9a, 4)] + v6 += [bytes(response[i:i + 16]) for i in range(0x9a, 0xea, 16)] + return ([a for a in dict.fromkeys(v4) if any(a)], + [a for a in dict.fromkeys(v6) if any(a)]) + + def check_apn(apn): """An APN as a carrier publishes it: labels of letters, digits and '-', joined by dots, at most 99 characters (the request's field is 100 bytes).""" @@ -84,8 +109,13 @@ def private_write(path, data): out.write(data) -def setup_payload(cid, radio_technology, pdu_session, apn, *, ims=False, ip_type='ipv4v6'): +def setup_payload(cid, radio_technology, pdu_session, apn, *, ims=False, ip_type=None): # Absolute request offsets below include the 12-byte request header. + # ip_type None: IPv6 for IMS (what every IMS setup used until 0.3.19), + # IPv4v6 otherwise. For IMS, stock takes the PDP type from the IMS APN's + # protocol and the P-CSCF request from it (GetPcscfReqType). + if ip_type is None: + ip_type = 'ipv6' if ims else 'ipv4v6' check_apn(apn) full = bytearray(983) full[12] = cid @@ -95,8 +125,8 @@ def setup_payload(cid, radio_technology, pdu_session, apn, *, ims=False, ip_type encoded = apn.encode('ascii') full[16:16 + len(encoded)] = encoded full[0xd9] = 0 # no authentication - full[0xda] = IP_TYPES['ipv6'] if ims else IP_TYPES[ip_type] - full[0xdb] = 2 if ims else 0 # IPv6 P-CSCF request + full[0xda] = IP_TYPES[ip_type] + full[0xdb] = PCSCF_REQUEST[ip_type] if ims else 0 # P-CSCF request (IPv6 2, both 3) # No handover addresses, roaming override or slice. # 0xf1 (u32): the APN-type bitmap, as stock's builder copies it from # ApnSetting's supported types (vendor.radio.base @@ -115,14 +145,28 @@ def setup_payload(cid, radio_technology, pdu_session, apn, *, ims=False, ip_type return bytes(full[12:]) -def initial_attach(client, apn, ip_type='ipv4v6'): +# The P-CSCF request type, as stock's GetPcscfReqType maps the APN protocol: +# IP 1, IPV6 2, IPV4V6 3. +PCSCF_REQUEST = {'ipv4': 1, 'ipv6': 2, 'ipv4v6': 3} + + +def initial_attach(client, apn, ip_type='ipv4v6', *, ims=False): """SET_INITIAL_ATTACH_APN (251 bytes). Without it, setup requests on this - phone returned status 36 and no bearer.""" + phone returned status 36 and no bearer. + + ims: the attach APN is the IMS APN (a network that attaches only there). + Stock's builder then writes the APN type at 0x0f and asks for the P-CSCF + at 0xdb (home) and 0xf9 (roaming) with GetPcscfReqType, so the network + returns its P-CSCF addresses in the attach bearer's PCO; without the + request, IMS on that bearer has none.""" attach = bytearray(251) attach[12:245] = setup_payload(1, 14, 0, apn, ip_type=ip_type)[:233] # Stock's attach builder leaves 0xf1-0xf4 zero: no APN-type bitmap here. attach[0xf1:0xf5] = bytes(4) - attach[0xf8] = 3 # the roaming PDP type, IPv4v6 (the home type is 0xda) + attach[0xf8] = IP_TYPES[ip_type] # the roaming PDP type (the home type is 0xda) + if ims: + attach[0x0f] = 1 # IMS APN type, as in the IMS setup request + attach[0xdb] = attach[0xf9] = PCSCF_REQUEST[ip_type] client.request(0x0603, bytes(attach[12:]), timeout=30) @@ -136,20 +180,137 @@ def allow_data(client): print('PS service permission enabled', flush=True) -def provision_default_profile(client, apn, ip_type='ipv4v6'): - # Stock BuildSetDataProfile, 246-byte request. This configures the - # network APN profile, not a SIM subscription or UICC operation. - encoded = check_apn(apn).encode('ascii') +# SET_DATA_PROFILE's APN-type bitmask (Android's ApnSetting TYPE_*_BITMASK). +APN_TYPES = {'default': 1, 'mms': 2, 'supl': 4, 'dun': 8, 'hipri': 16, 'fota': 32, + 'ims': 64, 'cbs': 128, 'ia': 256, 'emergency': 512, 'mcx': 1024, 'xcap': 2048} +# Android's profile IDs: 0 default (internet), 1 tethered, 2 IMS, 3 FOTA, +# 4 CBS; OEM profiles start at 1000. +PROFILE_IDS = 'a profile ID 0-4, or 1000 and up' +PROFILE_KEYS = ('types', 'ip', 'roaming_ip') + + +class Profile: + """One modem-managed data profile, as stock's BuildSetDataProfile sends it. + The defaults are stock's: 20 connections in 300 s, no wait, enabled, any + bearer, no authentication.""" + + def __init__(self, ident, apn, types, ip='ipv4v6', roaming_ip=None, *, + max_conns=20, max_conns_time=300, wait_time=0): + if not (0 <= ident <= 4 or 1000 <= ident <= 0x7fffffff): + raise ValueError(f'profile ID {ident}: not {PROFILE_IDS}') + if not types or types & ~sum(APN_TYPES.values()): + raise ValueError(f'profile {ident}: invalid APN types') + roaming_ip = roaming_ip or ip + for value in (ip, roaming_ip): + if value not in IP_TYPES: + raise ValueError(f'profile {ident}: IP type {value!r} is not ipv4, ipv6 or ipv4v6') + self.ident, self.apn, self.types = ident, check_apn(apn), types + self.ip, self.roaming_ip = ip, roaming_ip + self.max_conns, self.max_conns_time, self.wait_time = max_conns, max_conns_time, wait_time + + def __eq__(self, other): + return isinstance(other, Profile) and vars(self) == vars(other) + + def __repr__(self): + return f'Profile({self.ident}, {self.apn!r}, types=0x{self.types:x})' + + def type_names(self): + return ','.join(n for n, bit in APN_TYPES.items() if self.types & bit) + + +def default_profile(apn, ip_type='ipv4v6'): + """Profile 0 from [network] apn, byte for byte what 0.3.14 and earlier + sent (the connection limits and wait left 0): the default profile that + registered and carried data on both phones.""" + return Profile(0, apn, APN_TYPES['default'], ip_type, max_conns=0, max_conns_time=0) + + +def parse_profile(key, value): + """A [profiles] line: "ID = APN; types=a,b; ip=TYPE; roaming_ip=TYPE". + types is required; ip defaults to ipv4v6 and roaming_ip to ip.""" + try: + ident = int(key.strip(), 10) + except ValueError: + raise ValueError(f'profile {key!r}: the key must be {PROFILE_IDS}') from None + apn, *fields = [part.strip() for part in value.split(';')] + options = {} + for field in fields: + name, sep, item = (x.strip() for x in field.partition('=')) + name = name.lower() + if not sep or name not in PROFILE_KEYS or not item: + raise ValueError(f'profile {ident}: {field!r} is not one of ' + + ', '.join(k + '=...' for k in PROFILE_KEYS)) + if name in options: + raise ValueError(f'profile {ident}: {name} given twice') + options[name] = item.lower() + if 'types' not in options: + raise ValueError(f'profile {ident}: types=... is required') + types = 0 + for name in options.pop('types').split(','): + name = name.strip() + if name not in APN_TYPES: + raise ValueError(f'profile {ident}: unknown APN type {name!r}') + types |= APN_TYPES[name] + try: + check_apn(apn) + except ValueError: + raise ValueError(f'profile {ident}: the APN is missing or not a plain carrier APN') from None + return Profile(ident, apn, types, **options) + + +def configured_profiles(settings): + """The profiles to send, in profile ID order: [profiles] as written, and + profile 0 from [network] apn unless [profiles] has its own 0. Raises + ValueError on any bad line (start-up then stops before the radio).""" + net = settings['network'] + found = {} + if settings.has_section('profiles'): + for key, value in settings.items('profiles', raw=True): + if key in settings.defaults(): + continue + profile = parse_profile(key, value) + if profile.ident in found: + raise ValueError(f'profile {profile.ident} given twice') + found[profile.ident] = profile + if 0 not in found: + found[0] = default_profile(net.get('apn', ''), net.get('ip_type', 'ipv4v6')) + return [found[i] for i in sorted(found)] + + +def profile_payload(profile): + """Stock BuildSetDataProfile (V2_0), the 246-byte request. Offsets below + include the 12-byte request header. This configures a network APN + profile, not a SIM subscription or UICC operation.""" full = bytearray(246) - struct.pack_into(' 0x84 else None, + 'P-CSCF count IPv4', len(v4), 'IPv6', len(v6), flush=True) finally: if released: state_path.unlink(missing_ok=True) @@ -303,9 +465,16 @@ def deactivate(client, cid=1): owned = json.loads(state_path.read_text()) if owned.get('cid') != cid or not 0 <= owned.get('session', -1) <= 15: raise RuntimeError('invalid bearer ownership record') - flags = int(Path(f"/sys/class/net/rmnet{owned['cid'] - 1}/flags").read_text(), 16) - if flags & 1: - raise RuntimeError('remove host data configuration first') + # rmnet, and the device omamobile-sit-data found carrying it + # (data_network.find_device), while that configuration exists. + devices = {f"rmnet{owned['cid'] - 1}"} + recorded = RUN / 'data-interface' + if cid == 1 and recorded.exists(): + devices.add(recorded.read_text().strip()) + for dev in devices: + flags = Path(f'/sys/class/net/{dev}/flags') + if flags.exists() and int(flags.read_text(), 16) & 1: + raise RuntimeError('remove host data configuration first') # Stock BuildDeactDataCall: CID and normal-release reason. client.request(0x0601, bytes([owned['cid'], 0]), timeout=30) calls = call_ids(client.request(0x0602)) diff --git a/packages/omamobile-modem-sit/lib/data_network.py b/packages/omamobile-modem-sit/lib/data_network.py index 5500b61f1..68d8d3b9d 100644 --- a/packages/omamobile-modem-sit/lib/data_network.py +++ b/packages/omamobile-modem-sit/lib/data_network.py @@ -3,12 +3,21 @@ """omamobile-sit-data: the internet bearer (CID 1) as a fallback behind Wi-Fi. CPIF's rmnet devices are raw-IP links that NetworkManager doesn't manage -(the package marks rmnet* unmanaged). This owns only rmnet0's /32 address, -a metric-700 default route (Wi-Fi's default is 600, so Wi-Fi wins while it -is up) and the carrier's DNS while rmnet0 is the only way out. All state and -rollback records stay in RAM; nothing here is logged with an address. - -The carrier's DNS is used only while rmnet0 is the only way out. A bearer +(the package marks rmnet* unmanaged). This owns only the bearer's rmnet +device: its /32 address, a metric-700 default route (Wi-Fi's default is 600, +so Wi-Fi wins while it is up) and the carrier's DNS while it is the only way +out. All state and rollback records stay in RAM; nothing here is logged with +an address. + +Which rmnet carries the bearer: stock's RIL names it rmnet +(PdpContext's "%s%d" with the RIL socket's ifprefix and ifstart), and that is +tried first. On a network that attaches on another APN than the internet one, +the attach bearer held rmnet0 and the CID 1 internet bearer came out on +rmnet1 (cheetah, 2026-11-02), so the rest are tried after it: the device +that answers a ping with the bearer's address is the one kept, and its name +is recorded in RAM (data-interface) for the bearer's release. + +The carrier's DNS is used only while the bearer is the only way out. A bearer that sends but receives nothing, and then answers no ping either, loses its default route, and gets it back as soon as nothing else is left: otherwise a dead bearer's route plus its DNS can outlive a short Wi-Fi outage, make @@ -30,6 +39,9 @@ from data_call import RESPONSE RESOLVER = Path('/etc/resolv.conf') +INTERFACE = paths.RUN / 'data-interface' +# rmnet0-rmnet7 are tried; CPIF creates 30, a phone uses a handful. +CANDIDATES = 8 def run(*args): @@ -53,7 +65,7 @@ def configuration(data): def counters(stats): - """(received bytes, sent packets) from `ip -j -s link` output for rmnet0.""" + """(received bytes, sent packets) from `ip -j -s link` output for the device.""" link = stats[0]['stats64'] return link['rx']['bytes'], link['tx']['packets'] @@ -76,10 +88,10 @@ def guarded(step): return None -def answers(): - """Whether the bearer carries anything: a ping out through rmnet0.""" +def answers(dev): + """Whether the bearer carries anything: a ping out through its device.""" for target in ('1.1.1.1', '8.8.8.8'): - p = subprocess.run(['ping', '-n', '-q', '-c', '1', '-W', '3', '-I', 'rmnet0', target], + p = subprocess.run(['ping', '-n', '-q', '-c', '1', '-W', '3', '-I', dev, target], capture_output=True, timeout=10) if p.returncode == 0: return True @@ -115,28 +127,112 @@ def restore(self): class ResolvedLink: - """The same with systemd-resolved: rmnet0's DNS servers are set once and + """The same with systemd-resolved: the device's DNS servers are set once and the link is the default DNS route only while cellular is the only way out.""" - def __init__(self, servers): - run('resolvectl', 'dns', 'rmnet0', *servers) + def __init__(self, servers, dev): + self.dev = dev + run('resolvectl', 'dns', dev, *servers) self.cellular = None def update(self, cellular, servers): if cellular != self.cellular: - run('resolvectl', 'default-route', 'rmnet0', 'yes' if cellular else 'no') + run('resolvectl', 'default-route', self.dev, 'yes' if cellular else 'no') self.cellular = cellular def restore(self): - subprocess.run(['resolvectl', 'revert', 'rmnet0'], capture_output=True, timeout=20) + subprocess.run(['resolvectl', 'revert', self.dev], capture_output=True, timeout=20) -def resolver_for(servers): +def resolver_for(servers, dev): target = str(RESOLVER.resolve()) if RESOLVER.exists() else '' if target.startswith('/run/systemd/resolve/'): - return ResolvedLink(servers) + return ResolvedLink(servers, dev) return Resolver(RESOLVER) +def candidates(cid): + """rmnet (stock's name for the bearer) first, then the others.""" + first = cid - 1 + return [f'rmnet{first}'] + [f'rmnet{i}' for i in range(CANDIDATES) if i != first] + + +def unused(dev): + """The device exists and nobody has configured it (down, no address).""" + try: + info = json.loads(run('ip', '-j', 'address', 'show', 'dev', dev))[0] + except (RuntimeError, ValueError, IndexError): + return False + return 'UP' not in info['flags'] and not info['addr_info'] + + +def put_back(dev, address=None): + """Undo a candidate's configuration, every step on its own and none + fatal: whatever was set before a failure or a stop goes. Only devices + this service records (data-interface) or is probing get here, and rmnet + devices carry nothing but this service's configuration.""" + steps = [('ip', 'route', 'del', 'default', 'dev', dev, 'metric', '700')] + steps.append(('ip', 'addr', 'del', address + '/32', 'dev', dev) if address + else ('ip', 'addr', 'flush', 'dev', dev)) + steps.append(('ip', 'link', 'set', 'dev', dev, 'down')) + for step in steps: + try: + subprocess.run(step, capture_output=True, timeout=20) + except (OSError, subprocess.SubprocessError): + pass + + +def clear_stale(): + """A device recorded by a run that ended without its cleanup (killed, or + stopped before 0.3.12 while probing): put it back before anything else.""" + try: + dev = INTERFACE.read_text().strip() + except OSError: + return + if dev.startswith('rmnet') and dev[5:].isdigit(): + put_back(dev) + print('Removed a configuration left on', dev, 'by an earlier run', flush=True) + INTERFACE.unlink(missing_ok=True) + + +def find_device(address, cid, probe=None, stopped=lambda: False): + """Bring each unused candidate up with the bearer's /32 and the metric-700 + default route until one carries it (ping -I binds to the device, so the + route through it is used even with Wi-Fi's better one present); that one + keeps its address and route. Every other one is put back as it was, also + when a step fails or the service is stopped in the middle (the record in + data-interface is written before the device is touched, so a run that is + killed outright is cleaned up by the next start).""" + probe = probe or answers + for dev in candidates(cid): + if stopped(): + break + if not unused(dev): + continue + paths.run_dir() + INTERFACE.write_text(dev + '\n') + kept = False + try: + run('ip', 'link', 'set', 'dev', dev, 'up') + run('ip', 'addr', 'add', address + '/32', 'dev', dev) + run('ip', 'route', 'add', 'default', 'dev', dev, 'src', address, 'metric', '700') + try: + kept = probe(dev) and not stopped() + except subprocess.SubprocessError: + kept = False + except RuntimeError: + kept = False + finally: + if not kept: + put_back(dev, address) + INTERFACE.unlink(missing_ok=True) + if kept: + print('Bearer CID', cid, 'carried by', dev, flush=True) + return dev + if stopped(): + raise InterruptedError('Stopped while looking for the bearer device') + raise RuntimeError('No rmnet device carries the internet bearer (no ping answered)') + + def wait_for_bearer(seconds): deadline = time.monotonic() + seconds while not RESPONSE.exists(): @@ -149,35 +245,34 @@ def main(): os.umask(0o077) lock = os.open(paths.RUN / 'data-network.lock', os.O_CREAT | os.O_RDWR, 0o600) fcntl.flock(lock, fcntl.LOCK_EX | fcntl.LOCK_NB) + clear_stale() wait_for_bearer(60) - address, dns = configuration(RESPONSE.read_bytes()) - info = json.loads(run('ip', '-j', 'address', 'show', 'dev', 'rmnet0'))[0] - if 'UP' in info['flags'] or info['addr_info']: - raise RuntimeError('rmnet0 already configured; refusing to replace it') - resolver = resolver_for(dns) + data = RESPONSE.read_bytes() + address, dns = configuration(data) stopping = False addressed = False + dev = None def stop(*_): nonlocal stopping stopping = True signal.signal(signal.SIGTERM, stop) signal.signal(signal.SIGINT, stop) + resolver = None try: - run('ip', 'link', 'set', 'dev', 'rmnet0', 'up') - run('ip', 'addr', 'add', address + '/32', 'dev', 'rmnet0') + dev = find_device(address, data[2], stopped=lambda: stopping) addressed = True - run('ip', 'route', 'add', 'default', 'dev', 'rmnet0', 'src', address, 'metric', '700') + resolver = resolver_for(dns, dev) print('Cellular fallback ready; Wi-Fi remains preferred', flush=True) last = None - seen = counters(json.loads(run('ip', '-j', '-s', 'link', 'show', 'dev', 'rmnet0'))) + seen = counters(json.loads(run('ip', '-j', '-s', 'link', 'show', 'dev', dev))) check_at = time.monotonic() + 30 def watch(): nonlocal last, seen, check_at routes = json.loads(run('ip', '-j', 'route', 'show', 'default')) - routed = any(r.get('dev') == 'rmnet0' for r in routes) - others = [r for r in routes if r.get('dev') != 'rmnet0'] + routed = any(r.get('dev') == dev for r in routes) + others = [r for r in routes if r.get('dev') != dev] # With Wi-Fi up but rated limited, the carrier's DNS would keep # NetworkManager's own check failing and Wi-Fi penalised. cellular = routed and not others @@ -186,14 +281,14 @@ def watch(): print('Default internet path', 'cellular' if cellular else 'Wi-Fi/other', flush=True) last = cellular if not routed and not others: - run('ip', 'route', 'add', 'default', 'dev', 'rmnet0', 'src', address, 'metric', '700') + run('ip', 'route', 'add', 'default', 'dev', dev, 'src', address, 'metric', '700') print('Nothing else leads out; the cellular default route is back', flush=True) elif time.monotonic() >= check_at: - now = counters(json.loads(run('ip', '-j', '-s', 'link', 'show', 'dev', 'rmnet0'))) - if looks_silent(routed, seen, now) and not answers(): - run('ip', 'route', 'del', 'default', 'dev', 'rmnet0', 'metric', '700') + now = counters(json.loads(run('ip', '-j', '-s', 'link', 'show', 'dev', dev))) + if looks_silent(routed, seen, now) and not answers(dev): + run('ip', 'route', 'del', 'default', 'dev', dev, 'metric', '700') print('Cellular bearer carries nothing; its default route is removed', flush=True) - now = counters(json.loads(run('ip', '-j', '-s', 'link', 'show', 'dev', 'rmnet0'))) + now = counters(json.loads(run('ip', '-j', '-s', 'link', 'show', 'dev', dev))) seen = now check_at = time.monotonic() + 30 failing = None @@ -205,14 +300,20 @@ def watch(): print('Network checks work again', flush=True) failing = error time.sleep(2 if error else .5) + except InterruptedError: + # Stopped while probing: find_device has put the candidate back. + pass finally: - resolver.restore() - routes = json.loads(run('ip', '-j', 'route', 'show', 'default')) - if any(r.get('dev') == 'rmnet0' and r.get('metric') == 700 for r in routes): - run('ip', 'route', 'del', 'default', 'dev', 'rmnet0', 'metric', '700') - if addressed: - run('ip', 'addr', 'del', address + '/32', 'dev', 'rmnet0') - run('ip', 'link', 'set', 'dev', 'rmnet0', 'down') + if resolver: + resolver.restore() + if dev: + routes = json.loads(run('ip', '-j', 'route', 'show', 'default')) + if any(r.get('dev') == dev and r.get('metric') == 700 for r in routes): + run('ip', 'route', 'del', 'default', 'dev', dev, 'metric', '700') + if addressed: + run('ip', 'addr', 'del', address + '/32', 'dev', dev) + run('ip', 'link', 'set', 'dev', dev, 'down') + INTERFACE.unlink(missing_ok=True) os.close(lock) print('Cellular fallback configuration removed', flush=True) diff --git a/packages/omamobile-modem-sit/lib/ims.py b/packages/omamobile-modem-sit/lib/ims.py index db8d02aab..dd72b77b9 100644 --- a/packages/omamobile-modem-sit/lib/ims.py +++ b/packages/omamobile-modem-sit/lib/ims.py @@ -6,15 +6,32 @@ modem's own IMS stack registers. This announces ordinary voice/SMS capability; it does not originate a call. """ +import re import json import struct import time -from data_call import paths, call_ids, private_write +from data_call import paths, call_ids, pcscf_addresses, private_write from paths import RUN REGISTRATION = RUN / 'ims-registration.json' +# IND_AIMS_REGISTRATION's state byte and status code, as the stock IMS +# service names them (RilIndRegistration.RegState, RegistrationReason's CP +# reason). Status 5 is a permanent failure: stock tears the IMS PDN down +# after it (isNeedPdnDeactivationReason) and doesn't retry on that PDN. +REG_STATES = {1: 'VoLTE not registered', 2: 'VoLTE registered', 3: 'VoLTE limited', + 4: 'E911 not registered', 5: 'E911 registered', 6: 'RCS not registered', + 7: 'RCS registered', 8: 'E911 pseudo-registered'} +REG_STATUS = {0: 'success', 1: 'feature mismatch', 2: 'network-triggered deregistration', + 3: 'no response from the network', 4: 'internal deregistration', + 5: 'registration failed', 6: 'failed with 500', 7: 'failed with 403', + 8: 'failed, CP retrying', 9: 'limited registration', 10: 'failed, retry after cell change', + 11: 'PLMN barred', 12: 'fresh registration starting', 13: 'deregistration failed, call ongoing', + 14: 'failed, start timer', 15: 'failed, retry after Wi-Fi reassociation', + 16: 'failed, retry after RAT change', 18: 'failed, silent retry for a call', + 254: 'no valid SIM', 255: 'unspecified', 512: 'retry after PLMN change'} + def record_registration(packet): if len(packet) < 20: @@ -22,6 +39,13 @@ def record_registration(packet): state, features, mask, status, uri_len, reason_len = struct.unpack_from(' 1000 or reason_len > 256 or len(packet) < 20 + uri_len + reason_len: raise ValueError('truncated IMS registration indication') + # The reason is the IMS stack's own words (a SIP or stack failure); the + # URI before it is the subscriber's identity and is never logged. + reason = packet[20 + uri_len:20 + uri_len + reason_len].decode('ascii', 'replace').strip('\0 ') + reason = re.sub(r'\d{7,}', '', re.sub(r'(sip|tel):\S+', '', reason)) + print(f'IMS registration indication: state {state} ({REG_STATES.get(state, "unknown")}), ' + f'features {features}, status {status} ({REG_STATUS.get(status, "unknown")})' + + (f', reason "{reason}"' if reason else ''), flush=True) if REGISTRATION.exists(): private_write(REGISTRATION, json.dumps({'state': state, 'features': features, 'feature_mask': mask, 'status': status}).encode()) @@ -45,26 +69,62 @@ def hidden_menu(): return struct.pack('