Output too large. Showing first 8,000 and last 32,000 characters. For full output see: /home/Messier82/.gemini/tmp/service-book/tool-outputs/session-2384e908-ef6c-46eb-9d09-552b13434ec4/run_shell_command_1779366235929_0.txt Output: # GMD Smoke Test Scenarios (Phase 3) ## Purpose This document defines the detailed scenarios for the E2E smoke test suite that will run on Gradle Managed Devices (GMD). These tests verify critical happy-path user journeys on a real Android OS, complementing the comprehensive Robolectric-based integration tests that cover feature-level correctness. Smoke tests are **not** intended to replace integration tests. They exist to catch issues that only surface on a real Android environment: actual Activity lifecycle, real filesystem access via SAF, real Compose rendering pipeline, and cross-screen navigation with real back-stack behavior. ## Scope & Constraints - **Runner:** `AndroidJUnit4` with `createAndroidComposeRule()` - **Device:** Gradle Managed Device — Pixel 2, API 33 (aosp) - **Execution time budget:** The entire smoke suite should complete in under 2 minutes on a single GMD instance - **No external dependencies:** Tests must not require network, Google Play Billing, or pre-existing device state - **Test file:** `app/src/androidTest/java/com/yorvana/SmokeTest.kt` — single canonical entry point (replaces the instrumented `app/src/androidTest/java/com/yorvana/MainActivityTest.kt`) - **Coverage baseline:** Output is captured as `app/coverage-baselines/gmd_smoke.ec` via the `generateGmdCoverage` Gradle task ## Test Design Principles 1. **Two state strategies** — the core happy path (S01–S06) runs as a single multi-step flow test that accumulates state within one Activity launch; all other scenarios (S07–S17) are independent tests that reset DataStore and vault state in `@Before` and use setup helpers to reach the required precondition. See "Shared State vs. Independent Tests" in Implementation Notes for rationale. 2. **Semantic selectors** — use `testTag`, `contentDescription`, and visible text; avoid index-based selectors 3. **Wait-for-idle** — use `waitUntil` / `waitForIdle` instead of fixed delays 4. **Minimal assertions** — verify the screen rendered and key elements are present; leave exhaustive validation to Robolectric tests --- ## Scenarios ### S01: First Launch — Setup Screen **Precondition:** Fresh install, no vault configured. **Flow:** 1. App launches to `SetupScreen` 2. Verify "Welcome" title text is displayed 3. Verify setup description text is displayed 4. Verify "Choose Folder" button is visible and enabled **Verifies:** App cold-starts successfully, correct entry point for first-time users, `MainActivity` resolves the start destination to `Setup`. --- ### S02: Vault Configuration & Sentry Opt-in **Precondition:** App is on `SetupScreen`. **Flow:** 1. Tap "Choose Folder" button — SAF folder picker launches 2. Select or create a folder in the picker 3. App moves to Setup Step 2 (Sentry opt-in) 4. Verify "Help improve the app" title and toggle are visible 5. Toggle the switch and tap "Complete Setup" 6. App navigates to `VehicleListScreen` ("My Garage") 7. Verify the "My Garage" title is displayed **Verifies:** SAF integration works, 2-step setup flow is functional, vault URI and Sentry preference are persisted, `YorvanaApplication.applyVaultUri()` initializes storage correctly. **Note:** Interacting with the system SAF picker requires UiAutomator (`UiDevice`). If SAF interaction proves flaky or overly complex, this scenario can be simplified to only verify the picker launches (intent is fired) and the remaining flow can be tested by pre-configuring the vault in a `@Before` method using test fixtures. --- ### S03: Add a Vehicle **Precondition:** Vault is configured, `VehicleListScreen` is displayed with empty state. **Flow:** 1. Tap the FAB / "Add Vehicle" button 2. `AddEditVehicleScreen` is displayed 3. Verify the "Add Vehicle" title is shown 4. Enter a nickname (e.g., "Test Car") in the Nickname field 5. Optionally fill Make (e.g., "Toyota") and Year (e.g., "2020") 6. Tap the "Save" button 7. App navigates back to `VehicleListScreen` 8. Verify "Test Car" appears in the vehicle list 9. Verify the empty state is no longer shown **Verifies:** Navigation to AddEditVehicle, form input works, vehicle is persisted to the vault filesystem, vehicle list updates on return, denormalized stats are initialized. --- ### S04: View Vehicle Detail **Precondition:** At least one vehicle ("Test Car") exists in the list. **Flow:** 1. Tap on "Test Car" in the vehicle list 2. `VehicleDetailScreen` is displayed 3. Verify "Test Car" appears as the vehicle name/title 4. Verify the empty records state ("No service records yet…") is shown 5. Verify the "Add Record" FAB / button is visible **Verifies:** Navigation from VehicleList to VehicleDetail with correct vehicle ID, vehicle data loads from storage, empty records state renders. --- ### S05: Add a Service Record **Precondition:** On `VehicleDetailScreen` for "Test Car" with no records. **Flow:** 1. Tap the "Add Record" FAB / button 2. `AddEditRecordScreen` is displayed 3. Verify the "Add Record" title is shown 4. Verify the date field is non-empty and contains a valid, parseable date (do not assert an exact "today" string — it can be flaky around midnight, timezone, or locale changes; instead compare against the same formatter the app uses, or simply assert the field is populated) 5. Enter an odometer reading (e.g., "50000") 6. Select a service category from the dropdown (e.g., "Oil Change") 7. Enter a performer (e.g., "Myself") 8. Optionally enter a cost (e.g., "49.99") 9. Tap the "Save" button 10. App navigates back to `VehicleDetailScreen` 11. Verify the new record appears in the records list (showing "Oil Change" and/or the date) 12. Verify the empty records state is no longer shown **Verifies:** Navigation to AddEditRecord, form fields render with defaults, category selector works, record is persisted to vault filesystem, vehicle detail updates on return. --- ### S06: View Record Detail **Precondition:** On `VehicleDetailScreen` with at least one record. **Flow:** 1. Tap on the service record in the list 2. `RecordDetailScreen` is displayed 3. Verify the record date is shown 4. Verify the odometer reading is displayed — assert on stable pieces (e.g., the raw number via substring/regex and the unit separately) rather than an exact formatted string like "50,000 km", since formatting is locale-dependent (thousands separator, spacing). Set the locale and unit deterministically in test setup if an exact match is needed. 5. Verify the category ("Oil Change") is displayed 6. Verify the performer ("Myself") is displayed 7. Verify the cost is displayed (if entered) **Verifies:** Navigation from VehicleDetail to RecordDetail, record data loads correctly from storage, all fields render. --- ### S07: Add an Attachment via File Picker **Precondition:** On `AddEditRecordScreen` (either creating a new record or editing an existing one). **Flow:** 1. Scroll to the "Attachments" section 2. Verify the "Choose file" button is visible 3. Verify the "Take photo" button is visible 4. Tap the "Choose file" button — system file picker launches 5. Select a file (e.g., a test image or PDF pre-pushed to the device) 6. Verify the attachment thumbnail/entry appears in the Attachments section with the filename 7. Tap "Save" 8. App navigates back to the previous screen **Verifies:** File picker intent launches correctly, picked file URI is resolved and copied into the vault, attachment metadata is persisted with the record. **Note:** Selecting a file in the system picker requires UiAutomator (`UiDevice`). A test fixture file should be pre-pushed to the emulator's Downloads folder via an `adb push` step in the test setup or Gradle task. If picker interaction proves flaky, verify the intent fires and test the rest of the attachment flow by injecting a URI directly via the ViewModel event. --- ### S08: View Attachment on Record Detail **Precondition:** A record with at least one attachment exists (created in S07 or set up via test fixture). **Flow:** 1. Navigate to ... [6,504 characters omitted] ... wall-status-card"`); verify the "Learn more" action is present in the card. 3. Attempt swipe-to-dismiss on a vehicle row; verify the vehicle is still present (swipe-delete disabled in read-only). 4. Tap "Car A" → `VehicleDetailScreen`. Verify the status card is present; verify the "Add Record" FAB is visible but disabled (assert `stateDescription`). 5. Tap the Settings icon → `SettingsScreen`. Verify the "Premium" section shows the "Upgrade to Premium" entry. **Verifies:** `PaywallStatusCard` correctly signals read-only state on `VehicleListScreen`; `AppGate` propagates `isReadOnly` through all ViewModels; write-action UI is suppressed or disabled. --- ### S20: Reactive Lift on Premium Activation (Paywall) **Precondition:** Vault configured; 2 vehicles present; `setBillingOverride(FORCE_FREE)`. **Flow:** 1. Launch on `VehicleListScreen`. Verify the read-only status card is displayed. 2. From the test process, call `setBillingOverride(FORCE_PREMIUM)`. 3. `waitUntil` the status card is no longer displayed (timeout 5s — involves DataStore write + Flow combine + recomposition). 4. Tap the "Add Vehicle" FAB and verify it navigates to `AddEditVehicleScreen` (no upgrade sheet). 5. Press back; tap "Car A" → `VehicleDetailScreen`. Verify the status card is absent and the "Add Record" FAB is enabled. **Verifies:** Reactive lift of read-only mode end-to-end through `BillingManagerImpl` → `AppGate` → ViewModels → UI on the real Compose pipeline. Catches stale-state bugs in the gate's `combine` pipeline that unit tests can miss when scope/dispatcher choices differ from production. --- ### S17: Back Navigation **Precondition:** App has navigated several screens deep (e.g., VehicleList -> VehicleDetail -> RecordDetail). **Flow:** 1. Starting from `RecordDetailScreen` 2. Press the system back button or toolbar back arrow 3. Verify `VehicleDetailScreen` is displayed 4. Press back again 5. Verify `VehicleListScreen` is displayed **Verifies:** Back-stack is maintained correctly through the real Activity lifecycle, navigation transitions render without crashes. --- ### S21: Crash Reporting Opt-in Persists (Sentry) **Precondition:** Vault configured. **Flow:** 1. Navigate to `SettingsScreen`. 2. Find the "Help improve the app" toggle and turn it **ON**. 3. Exit the app (or recreate the Activity). 4. Navigate back to `SettingsScreen`. 5. Verify the toggle is still **ON**. 6. Turn the toggle **OFF**. 7. Exit the app (or recreate the Activity). 8. Navigate back to `SettingsScreen`. 9. Verify the toggle is still **OFF**. **Verifies:** Crash reporting preference persists across sessions via DataStore and correctly reflects in the UI. This protects the plumbing that determines if Sentry initializes on next launch. --- ### S22: Report Bug Screen Rendering **Precondition:** App is on `SettingsScreen`. **Flow:** 1. Tap on "Report a bug or suggest a feature". 2. `ReportBugScreen` is displayed. 3. Verify the "Feedback" title is shown. 4. Verify the Summary and Description fields are visible. 5. Tap the "Device information" card and verify it expands to show technical data. 6. Verify the Submit button is present and initially disabled. **Verifies:** Navigation from Settings to ReportBugScreen, expressive UI components render correctly, and form validation logic is active. --- ## Scenarios Explicitly Out of Scope The following are **not** covered by smoke tests and are handled by Robolectric integration tests or manual testing: | Area | Reason | |---|---| | **Form validation errors** (empty required fields, VIN duplicates, character limits) | Covered exhaustively by ViewModel unit tests and screen integration tests | | **Image viewer gestures** (pinch-zoom, pan, double-tap) | Gesture simulation on GMD is unreliable; covered by screenshot tests and Robolectric. Navigation to ImageViewer is covered by S08 | | **Real Google Play Billing purchase sheet** | Requires the device to be signed into a license-tester account with the app on a Play track. Covered manually per `docs/plan-paywall.md` Phase 8 Track B. | | **Odometer unit / currency switching** | Settings persistence covered by unit tests; display rendering covered by Robolectric | | **Data folder relocation** | SAF folder picker interaction; high complexity for smoke tests | | **Edge cases** (200+ records, 20 vehicles, large attachments) | Performance testing, not smoke testing | | **Error states** (corrupted JSON, missing files, permission denied) | Defensive behavior covered by unit tests per ADR-006 | --- ## Phased Implementation Plan ### Step 1: Test Infrastructure & Helpers **Goal:** Set up the `SmokeTest.kt` file with all necessary test infrastructure before writing scenario logic. 1. **Rename entry point:** Rename `MainActivityTest.kt` to `SmokeTest.kt` (or create `SmokeTest.kt` and delete `MainActivityTest.kt`). Update the class name to `SmokeTest`. Keep the `@RunWith(AndroidJUnit4::class)` runner and `createAndroidComposeRule()` rule. 2. **Add UiAutomator dependency:** Add `androidx.test.uiautomator:uiautomator` to `androidTestImplementation` in `app/build.gradle.kts` for system UI interactions (SAF picker, camera). 3. **Add test ordering:** Annotate the class with `@FixMethodOrder(MethodSorters.NAME_ASCENDING)` so scenarios execute in `test_01_`, `test_02_` order. 4. **Vault setup helper:** Create a helper method `configureVaultForTesting()` that creates a temporary vault directory on the emulator's internal storage, persists its URI through the app's DataStore-backed preferences (for example via `AppPreferencesStore.setVaultUri(...)`, or by editing the DataStore directly), and then calls `YorvanaApplication.applyVaultUri()` to bypass the SAF picker for most tests. 5. **Navigation helpers:** Create small helper methods for common navigation steps — `navigateToVehicleDetail(name)`, `navigateToAddRecord()`, `navigateToSettings()` — using Compose test semantics (`onNodeWithText`, `performClick`). These reduce duplication across scenarios. 6. **Attachment fixtures:** Create `app/src/androidTest/assets/` directory with a small test image (`test_receipt.jpg`, ~10KB) and a test PDF (`test_invoice.pdf`, ~5KB). Add a `@Before` or helper that copies these into a picker-visible location for SAF simulation — on API 33, prefer inserting them into Downloads via `MediaStore` (with unique test-specific names) rather than using an app-private directory such as `context.cacheDir`, which DocumentsUI generally cannot browse. 7. **Cleanup rule:** Add a `@After` (or `TestWatcher` rule) that wipes the temporary vault directory and deletes any test fixture entries created in the picker-visible location (for example, the Downloads `MediaStore` items created for the run) to prevent cross-run pollution. 8. **Billing override helper (added with paywall feature, see `docs/plan-paywall.md` Phase 7.5):** Add `setBillingOverride(mode: DebugBillingOverride.Mode)` that calls `(application as YorvanaApplication).debugBillingOverride.setMode(mode)` from the test process. Add an `@After` rule that resets the override to `NONE` to prevent state leaking across scenarios. Add `addVehicleForTesting(nickname)` helper that persists a vehicle directly via `VehicleRepository`, so paywall scenarios can start from a known multi-vehicle state without depending on S03 having run. 9. **Local verification:** Run `./gradlew pixel2api33DebugAndroidTest` to confirm the empty test class compiles, deploys, and the infrastructure helpers initialize without errors. **Deliverables:** `SmokeTest.kt` with infrastructure, helpers, fixtures, and cleanup — no scenario logic yet. `MainActivityTest.kt` removed. --- ### Step 2: Core Happy Path (S01–S06) **Goal:** Implement the primary user journey — from first launch through viewing a record — as a single multi-step flow test. 1. **S01 — First launch:** In `test_01_coreHappyPath_flow()`, start from first launch and assert "Welcome" text and "Choose Folder" button are displayed. 2. **S02 — Vault configuration:** In the same flow test, use the hybrid approach: verify the picker intent fires on button tap, then call `configureVaultForTesting()` programmatically and navigate to `VehicleListScreen`. Assert empty state is shown. 3. **S03 — Add vehicle:** Continue in the same flow test. Navigate to AddEditVehicle, fill the nickname field, save, and assert the vehicle appears in the list. 4. **S04 — View vehicle detail:** Continue in the same flow test. Tap the vehicle card and assert the detail screen renders with the vehicle name and empty records state. 5. **S05 — Add service record:** Continue in the same flow test. Navigate to AddEditRecord, fill odometer/category/performer, save, and assert the record appears in the vehicle detail list. 6. **S06 — View record detail:** Finish the same flow test by tapping the record and asserting all fields render on `RecordDetailScreen`. 7. **Local verification:** Run the full suite on GMD. Ensure the single S01–S06 flow test passes reliably end-to-end. Address any timing issues with `waitUntil` / `waitForIdle`. **Deliverables:** 1 passing multi-step flow test covering the core CRUD happy path from S01 through S06. This is the minimum viable smoke suite. --- ### Step 3: Attachment Scenarios (S07–S10) **Goal:** Add attachment lifecycle coverage — the primary reason these tests must run on a real device. 1. **S07 — Add attachment via file picker:** Implement `test_07_addAttachmentViaFilePicker()`. Start by creating or editing a record. Use UiAutomator to interact with the system file picker (or use `ActivityResultLauncher` test doubles via `ActivityScenario` if UiAutomator is too fragile). Select the fixture image from the known device path. Assert the attachment thumbnail appears. Save the record. 2. **S08 — View attachment on record detail:** Implement `test_08_viewAttachmentOnRecordDetail()`. Navigate to RecordDetailScreen for the record with attachments. Assert the "Attachments (1)" header and attachment card are displayed. Tap the image attachment, assert `ImageViewerScreen` is displayed, press back. 3. **S09 — Remove attachment:** Implement `test_09_removeAttachment()`. Edit the record, tap "Remove" on the attachment, confirm the dialog, assert the attachment is gone, save. Navigate to RecordDetail and verify the Attachments section is absent or empty. 4. **S10 — Take photo attachment:** Implement `test_10_takePhotoAttachment()`. On AddEditRecordScreen, tap "Take photo", interact with the emulated camera (or stub the camera result). Assert the photo thumbnail appears. Save. 5. **Flakiness assessment:** Run the attachment tests 5 times locally. If S10 (camera) fails >20% of the time, downgrade it to a "picker launch verification only" test and document the limitation. 6. **Local verification:** Full suite (S01–S10) passes on GMD. **Deliverables:** 4 attachment tests. Camera test may be marked as best-effort depending on emulator reliability. --- ### Step 4: Navigation & Settings (S11–S12, S17) **Goal:** Cover settings screens and back-stack navigation. 1. **S11 — Navigate to Settings:** Implement `test_11_navigateToSettings()`. From VehicleListScreen, tap Settings, assert the settings screen renders with odometer unit, currency, and vault location. 2. **S12 — Navigate to Categories:** Implement `test_12_navigateToCategories()`. From SettingsScreen, tap Categories, assert predefined categories are listed, navigate back. 3. **S17 — Back navigation:** Implement `test_17_backNavigation()`. Navigate VehicleList → VehicleDetail → RecordDetail, then press back twice, asserting each intermediate screen. 4. **Local verification:** Full suite passes. **Deliverables:** 3 tests covering settings/categories screens and back-stack integrity. --- ### Step 5: Edit & Delete Flows (S13–S16) **Goal:** Cover mutation and deletion paths with confirmation guards. 1. **S13 — Edit vehicle:** Implement `test_13_editVehicle()`. Navigate to VehicleDetail, tap Edit, verify pre-filled nickname, change it, save, assert updated name on VehicleDetail. 2. **S14 — Edit service record:** Implement `test_14_editServiceRecord()`. Navigate to RecordDetail, tap Edit, verify pre-filled odometer, change it, save, assert updated value on RecordDetail. 3. **S15 — Delete service record:** Implement `test_15_deleteServiceRecord()`. On RecordDetail, tap Delete, confirm dialog, assert return to VehicleDetail with record removed. 4. **S16 — Delete vehicle:** Implement `test_16_deleteVehicle()`. Initiate vehicle deletion, confirm dialog, assert return to VehicleList with vehicle removed and empty state shown. 5. **Ordering consideration:** S15 and S16 are destructive. If using shared state (approach 1), these must run last. If the test data from S03/S05 was already consumed by prior tests, create fresh test data in a setup helper before running the delete scenarios. 6. **Local verification:** Full suite (S01–S17) passes on GMD. **Deliverables:** 4 tests covering edit and delete flows. Complete smoke suite of 17 scenarios. --- ### Step 6: Paywall Scenarios (S18–S20) **Goal:** Cover paywall gate, read-only banner propagation, and reactive lift on premium activation. Depends on `DebugBillingOverride` shipping with the paywall feature (`docs/plan-paywall.md` Phase 3.3) — sequence this step **after** that work. 1. **S18 — Upgrade dialog on free tier:** Implement `test_18_upgradeDialogOnFreeTier()`. Setup: 1 vehicle + `FORCE_FREE`. Tap FAB; assert dialog renders; dismiss; assert dialog gone. 2. **S19 — Read-only banner across screens:** Implement `test_19_readOnlyBannerAcrossScreens()`. Setup: 2 vehicles + `FORCE_FREE`. Visit VehicleList → VehicleDetail → Settings; assert banner present on each; assert write actions suppressed. 3. **S20 — Read-only lifts on premium activation:** Implement `test_20_readOnlyLiftsOnPremium()`. Setup: 2 vehicles + `FORCE_FREE`. Switch override to `FORCE_PREMIUM` from the test process; `waitUntil` banner disappears (timeout 5s); assert FAB now navigates to AddVehicle. 4. **Override reset:** Confirm `@After` resets override to `NONE` so subsequent tests are unaffected. 5. **Local verification:** Full suite (S01–S20) passes on GMD; verify paywall tests do not contaminate non-paywall scenarios. **Deliverables:** 3 paywall smoke tests. Suite total: 20 scenarios. --- ### Step 7: Coverage Baseline & Final Verification **Goal:** Generate the coverage baseline and validate the complete suite. 1. **Full local run:** Run `./gradlew pixel2api33DebugAndroidTest` and confirm all 17 tests pass. 2. **Timing check:** Verify the entire suite completes within the 2-minute budget. If it exceeds the budget, identify the slowest scenarios and consider merging related scenarios (e.g., combine S11+S12 into a single settings navigation test). 3. **Generate coverage:** Run `./gradlew generateGmdCoverage` to produce the updated `gmd_smoke.ec` baseline. 4. **Coverage report:** Run `./gradlew verifyWithCoverage` and review the report. Confirm the smoke tests contribute meaningful coverage to areas not covered by Robolectric (Activity lifecycle, SAF integration, real filesystem paths). 5. **Commit:** Commit `SmokeTest.kt`, attachment fixtures, dependency changes, and the updated `gmd_smoke.ec` to Git LFS. 6. **Update documentation:** Update `TESTING_SETUP.md` to document the smoke test suite, its scenarios, and how to run/maintain it. **Deliverables:** Final smoke suite committed, coverage baseline refreshed, documentation updated. Phase 3 complete. --- ## Implementation Notes ### Shared State vs. Independent Tests Two viable strategies: 1. **Single flow test** — implement the core happy-path journey as one `@Test` method with multiple sequential steps. With `createAndroidComposeRule()`, this is the only approach that truly keeps a single Activity launch for the entire chain. Fastest for the end-to-end path, but failures stop the remainder of that flow. 2. **Independent tests with setup helpers** — each `@Test` launches a fresh `MainActivity` and uses helper methods to reach the required precondition state. If tests are ordered, they may still share persisted state such as vault/files on disk, but they do **not** share one in-memory Activity instance. Slower (multiple launches) but more robust. **Recommendation:** Use approach 1 for the core happy-path chain (S01-S06) as a single multi-step flow test, and approach 2 for isolated scenarios (S07-S17) that should remain independent of prior in-memory UI state. ### SAF Picker Interaction The SAF folder picker (S02) is a system UI outside the app's Compose tree. Options: - **UiAutomator** (`UiDevice.findObject()`) to interact with the system picker — most realistic but fragile across API levels - **Pre-configure vault** in a `@Before` / test fixture by writing the vault URI via DataStore (for example, `AppPreferencesStore.setVaultUri`, backed by `Context.dataStore`) and then calling `YorvanaApplication.applyVaultUri()` — skips SAF but tests everything after it - **Hybrid** — one dedicated test for SAF launch verification, remaining tests use the DataStore-preconfigured vault **Recommendation:** Use the hybrid approach. One test verifies the picker launches; all others pre-configure the vault via DataStore. ### Attachment Test Fixtures Attachment scenarios (S07-S10) require files to be available on the emulator. Options: - **Pre-push via Gradle:** Add a task that runs `adb push` to place a small test image (e.g., `test_receipt.jpg`) and a test PDF into the emulator's Downloads folder before tests execute - **Bundle in `androidTest/assets`:** Place fixture files in `app/src/androidTest/assets/` and copy them to a known device path in `@Before` - **Programmatic creation:** Generate a minimal bitmap or text file in `@Before` and write it to the device filesystem **Recommendation:** Bundle fixtures in `androidTest/assets` — this is self-contained, doesn't require `adb`, and works reliably on GMD. ### Test Data Cleanup Tests that create vehicles/records should clean up after themselves or use a temporary vault directory that is wiped in `@After`. This prevents test pollution across runs and ensures `generateGmdCoverage` produces consistent results. # Yorvana — Testing Setup & Infrastructure This document summarizes the testing architecture and the steps required to re-establish the environment, especially when working within a Distrobox or containerized setup. ## 1. Test Organization (Naming Convention) To maintain a clear distinction between fast JVM tests and Android-dependent tests, we follow these naming rules: - **`*Test.kt`**: Unit Tests or Local UI Tests. Run on the local JVM (extremely fast). UI tests use **Robolectric** to simulate the Android environment. - **`*IT.kt`**: Local Integration Tests. Use **Robolectric** to simulate Android and test interactions between multiple components (e.g., Repositories + Storage). These are found in `src/test/java`. - **`app/src/androidTest/...`**: Instrumented Integration Tests. Run on a **real Android OS** (Emulator or Device). Used for high-fidelity verification of features that depend heavily on actual OS behavior. - **`*ScreenshotTest.kt`**: Visual regression tests using Roborazzi running on the JVM (Robolectric). Note: Historically, some `*IT.kt` files were located in `src/androidTest`, but we are migrating them to `src/test` to leverage Robolectric for faster feedback loops. New integration tests should be added to `src/test`. ## 2. Instrumented Tests via Gradle Managed Devices (GMD) We use GMD to automate emulator management. This avoids the need to manually create AVDs. ### Running Tests ```bash # Clean existing snapshots/devices (recommended after env changes or GPU mode changes) ./gradlew cleanManagedDevices # Run the instrumented tests on the managed Pixel 2 (API 33) ./gradlew pixel2api33Check ``` ### Configuration Details - **Smoke device**: Pixel 2, API Level 33. - **Store screenshot devices**: Pixel 6, Nexus 7, and Pixel Tablet, API Level 34. - **Image**: `aosp`. - **Location**: Defined in `app/build.gradle.kts` under `testOptions.managedDevices`. - **GPU mode**: Set via `android.testoptions.manageddevices.emulator.gpu` in `gradle.properties` (see below). --- ## 3. GPU Rendering Modes The emulator GPU mode controls how the Android guest renders graphics. Choosing the right mode depends on your environment. ### `host` — Hardware acceleration (host GPU passthrough) The emulator uses the host machine's physical GPU via Vulkan/gfxstream. | | | |---|---| | **Pros** | Fastest; required for screenshot tests to produce accurate pixel output | | **Cons** | Requires KVM + GPU passthrough; can hang the host if the GPU driver conflicts | **Works on:** - Bare metal Linux with KVM and a compatible GPU driver - Distrobox with KVM + GPU passthrough configured **Does NOT work reliably on:** - Distrobox without GPU passthrough - GitHub Actions (Ubuntu runners have no physical GPU) ### `swiftshader_indirect` — Software rendering (Google SwiftShader) The emulator uses Google's SwiftShader Vulkan ICD bundled with the Android SDK. No host GPU involved. | | | |---|---| | **Pros** | No GPU required; used by CI | | **Cons** | Crashes (SIGSEGV in QEMU) on Linux kernel ≥ 6.17 due to a conflict in the gfxstream `GLAsyncSwap` code path | **Works on:** - GitHub Actions (`ubuntu-latest` with KVM, kernel < 6.17) - Bare metal Linux with kernel < 6.17 **Does NOT work on:** - Linux kernel ≥ 6.17 (e.g. Fedora 43 / kernel 6.17): QEMU segfaults during cold boot ### `angle_indirect` — Software rendering (ANGLE + lavapipe/llvmpipe) The emulator treats this as an invalid option and falls back to `auto`, which selects Mesa's lavapipe (llvmpipe, LLVM-JIT software rasterizer) via ANGLE. Crucially, lavapipe disables `GLAsyncSwap`, which avoids the SIGSEGV present in the SwiftShader path. | | | |---|---| | **Pros** | Works on kernel ≥ 6.17; no GPU required; does not hang the host | | **Cons** | Slower than `host`; the fallback to lavapipe is an implementation detail of the emulator, not an officially documented mode | **Works on:** - Distrobox on Linux kernel ≥ 6.17 (current local setup) - Any Linux without a GPU --- ## 4. Current Configuration | Environment | `gradle.properties` setting | Effective renderer | Notes | |---|---|---|---| | **Local (Distrobox, kernel 6.17+)** | `angle_indirect` | lavapipe (llvmpipe via ANGLE) | Set in `gradle.properties` | | **CI (GitHub Actions, ubuntu-latest)** | `swiftshader_indirect` | SwiftShader | Overridden at runtime via `-Pandroid.testoptions.manageddevices.emulator.gpu=swiftshader_indirect` | | **Bare metal with GPU** | `host` | Host GPU (NVIDIA etc.) | Override `gradle.properties` locally | To run locally with hardware acceleration (e.g. on bare metal): ```bash ./gradlew cleanManagedDevices ./gradlew pixel2api33Check -Pandroid.testoptions.manageddevices.emulator.gpu=host ``` --- ## 5. Continuous Integration (CI) CI is configured in `.github/workflows/ci.yml`. Instrumented tests run on `ubuntu-latest` with KVM enabled: ```yaml - name: Enable KVM permissions run: sudo chmod 666 /dev/kvm - name: Run All Tests & Generate Combined Coverage run: ./gradlew verifyWithCoverage -Pandroid.testoptions.manageddevices.emulator.gpu=swiftshader_indirect -Proborazzi.test.verify=true -Pcoverage --info ``` CI uses `swiftshader_indirect` for software rendering because GitHub-hosted runners lack a physical GPU. Local executions will default to the `host` GPU (specified in `gradle.properties`) for maximum performance, but can be overridden using the same `-P` flag if hardware acceleration is unavailable. --- ## 6. Troubleshooting - **Segfault (Exit Code 139) with `host` GPU on Distrobox (NVIDIA)**: The Android Emulator may crash on startup due to incompatible Mesa Vulkan wrappers (e.g., `dzn_icd.json`). To fix this, explicitly point the emulator to the NVIDIA ICD. Add `export VK_ICD_FILENAMES=/usr/share/vulkan/icd.d/nvidia_icd.x86_64.json` to your `~/.bashrc` inside the Distrobox container. - **Segfault (Exit Code 139) with `swiftshader_indirect`**: Known crash on Linux kernel ≥ 6.17. Switch to `angle_indirect` in `gradle.properties`. - **Host machine hangs during tests with `host` GPU**: GPU driver conflict (gfxstream + NVIDIA via KVM). Switch to `angle_indirect` for software rendering. - **KVM Access**: Verify with `ls -l /dev/kvm`. It must be accessible to your user. - **Clean Slate**: If GMD gets into a weird state, always try `./gradlew cleanManagedDevices` before re-running. - **Changing GPU mode**: Always run `cleanManagedDevices` first — snapshots created under one GPU mode are not reusable under another. --- ## 7. Screenshot Testing (Roborazzi) Most screenshot tests use **Roborazzi** and are executed on the **JVM using Robolectric** rather than via Gradle Managed Devices (GMD). ### Why JVM instead of GMD? While GMD provides higher fidelity through hardware-accelerated rendering, it introduces two critical workflow blockers: 1. **Silent Failures:** Roborazzi running in GMD instrumentation fails to properly verify or assert missing snapshots, silently passing tests. 2. **Artifact Extraction:** The GMD emulator is ephemeral. It spins up, runs tests, writes snapshots to its internal storage, and tears down immediately. Pulling the newly recorded `*.png` files back to the host machine for commit becomes extremely difficult and manual. Running screenshot tests in `src/test` ensures snapshots are generated directly onto the host filesystem, providing a reliable and frictionless developer experience. The exception is the Play Store marketing suite in `app/src/androidTest/java/com/yorvana/screenshots/`. It is intentionally instrumented because store assets should use real platform rendering and native device dimensions. The suite captures PNGs from the emulator display into the managed device additional-output directory. ### High-Fidelity Screenshots (Release Build) By default, `generateStoreScreenshots` runs against the **debug** variant for speed and stability. To generate production-representative assets from the minified **release** build (hiding developer settings and matching production UI exactly), use the `-PscreenshotBuild` flag: ```bash ./gradlew generateStoreScreenshots -PscreenshotBuild ``` This task runs only the `com.yorvana.screenshots` package on the three API 34 store devices and copies the generated PNGs into `distribution/store-assets/screenshots/{phone,tablet-7,tablet-10}/`. The existing Pixel 2 smoke device is kept separate and excludes the screenshot package during smoke invocations. --- ## 8. Robolectric Configuration We use Robolectric to run Android-dependent tests (including UI and integration tests) on the JVM for faster execution and better developer experience. ### Graphics Mode By default, all Robolectric tests in this project inherit the **NATIVE** graphics mode. This is configured globally in `app/build.gradle.kts` via `systemProperty("robolectric.graphicsMode", "NATIVE")`. - **Native Graphics**: Required for Roborazzi screenshot tests and provides high-fidelity rendering for complex UI interactions. - **Overriding**: You only need to add `@GraphicsMode(GraphicsMode.Mode.NATIVE)` or `@GraphicsMode(GraphicsMode.Mode.LEGACY)` if a test specifically requires an override or for explicit documentation. ### Migration Checklist (androidTest to test) We prioritize running tests on the JVM via Robolectric. However, a test must remain in (or be added to) `src/androidTest` if it: - Uses `ActivityScenario`, `createAndroidComposeRule`, or otherwise requires a real instrumentation-backed `Activity`/scenario. (`createComposeRule()` is supported in `src/test` Robolectric/JVM tests.) - Depends on `UiDevice` or UiAutomator APIs for system-level interactions. - Interacts with the real filesystem via the Storage Access Framework (SAF) using a non-mocked `ContentResolver`, `DocumentsProvider`, or `DocumentFile`. - Requires a running `Instrumentation` instance (e.g., `InstrumentationRegistry.getInstrumentation()`). - Tests hardware-dependent behavior (camera, sensors, Bluetooth, etc.). If a test only *mocks* these dependencies (e.g., using `mockk()`), it is eligible for migration to the JVM. --- ## 9. Smoke Test Suite (GMD) The Smoke Test suite (`app/src/androidTest/java/com/yorvana/SmokeTest.kt`) performs end-to-end verification of critical happy-path journeys on a real Android OS using Gradle Managed Devices (GMD). ### Purpose Smoke tests catch issues that only surface on a real Android environment: - Actual Activity lifecycle behavior, plus Compose navigation, state restoration, and back-stack handling. - Real file-based vault I/O on device/emulator storage, plus attachment access through Android `ContentResolver` / `MediaStore` integration. - System-level intent flows (camera capture and file-picker interactions). - Cross-screen navigation and back-stack integrity. ### Scenarios Covered The suite covers 17 scenarios (S01–S17), including: - **Core Happy Path**: First launch, vault configuration, adding/viewing vehicles and records. - **Attachments**: Adding via file picker, viewing in the gallery, removing, and taking photos. - **Settings & Navigation**: Navigating to Settings/Categories and verifying back-stack behavior. - **Mutations**: Editing and deleting vehicles and records with confirmation guards. ### Running Smoke Tests ```bash # Run just the debug androidTest task on the device (fastest for local verification) ./gradlew pixel2api33DebugAndroidTest # Run the broader device check task ./gradlew pixel2api33Check # Run tests and refresh the coverage baseline (gmd_smoke.ec) ./gradlew generateGmdCoverage ``` ### Refreshing the coverage baseline When material changes land in `app/src/main` (UI flows, storage logic, ViewModels exercised by smoke tests), regenerate and commit the baseline: ```bash ./gradlew generateGmdCoverage git add app/coverage-baselines/gmd_smoke.ec git commit -m "chore: refresh smoke coverage baseline" ``` The `Baseline Reminder` workflow posts a sticky PR comment on any PR that touches `app/src/main/**`, prompting you to refresh the baseline if the changes affect smoke coverage. Note that the `Smoke` workflow itself is path-filtered to UI changes (`app/src/main/**/ui/**`) — non-UI production changes won't trigger smoke, but the reminder will still fire so coverage stays current. ### Coverage Integration Smoke test results are persisted as a static coverage baseline in `app/coverage-baselines/gmd_smoke.ec`. This baseline is combined with JVM unit test data during report generation: ```bash # Generate combined report (JVM tests + Smoke test baseline) ./gradlew verifyWithCoverage ``` Since `app/coverage-baselines/*.ec` is tracked via Git LFS, a fresh clone can contain only an LFS pointer; `verifyWithCoverage` will fail until `git lfs pull` is run. ### Troubleshooting **`verifyWithCoverage` fails with "Coverage baseline is a Git LFS pointer"** The baseline file in your working tree is the small LFS pointer instead of the real `.ec` content. Fix: ```bash git lfs pull ``` If you do not have Git LFS installed, install it (`sudo dnf install git-lfs` on Fedora, `brew install git-lfs` on macOS) and run `git lfs install` once before re-pulling. As a fallback you can regenerate the baseline locally with `./gradlew generateGmdCoverage` (requires the GMD emulator to run). **`verifyWithCoverage` fails with "No dynamic JaCoCo execution data files were found"** The JVM unit-test execution data is missing. Re-run with coverage enabled: ```bash ./gradlew testDebugUnitTest -Pcoverage ./gradlew verifyWithCoverage ``` **Smoke tests fail with "Activity opened at unexpected destination"** Stale DataStore state from a previous run. The `ClearAppStateRule` resets this before each test, but if you see it during local debugging, run `./gradlew cleanManagedDevices` before retrying. Process Group PGID: 465590