# earthUI 0.8.0

## Per-project settings: save / restore

* Model configuration is now keyed by **(project, purpose)** — the data
  file is no longer part of the key. A project's several input files (a
  small test extract, the full dataset) share one variable/parameter
  configuration, so switching files keeps the setup. `get_project_settings()`
  and `set_project_settings()` lost their `file_basename` argument
  accordingly.
* **Button-only save.** The "Save current as default" buttons are the save
  mechanism: the **section 3** button saves the target, effective date, and
  predictor configuration; the **section 4** button saves the earth call
  parameters and the Allowed Interactions matrix. Each button updates only
  its own half (read-merge), so they never clobber each other. **Fit no
  longer auto-saves** — you can fit experimentally without overwriting your
  saved default. There is no on-change auto-save.
* Save and restore are **server-side and DB-authoritative**: configuration
  is written to / read from `projects.sqlite` directly from the inputs at the
  render and click, with no localStorage round-trip. This removes a class of
  restore races and a JSON double-encoding bug that could corrupt a saved row
  on the next save.
* The **Effective Date** field is shown in all purposes (general/CMA and
  market values have an effective date too, not just USPAP/IVS appraisals);
  it is still only used to compute `sale_age` in appraisal/market modes.

## Database durability

* `projects.sqlite` opens in **WAL** journaling mode where supported, for
  crash-safe writes and non-blocking reads.
* The settings-table schema **migration runs in a single transaction** with
  crash recovery: an interrupted upgrade rolls back to the original table
  (no orphaned `_old`, no half-copied data), and a previously-stranded
  `_old` table is restored on the next connect. Legacy per-file rows are
  collapsed to one row per (project, purpose), keeping the most recent.

## Bug fixes

* Predicting on raw data with a **Date column as a model predictor** no
  longer errors ("subscript out of bounds" / column-count mismatch).
  Newdata is now aligned to the trained frame (`align_to_training_`):
  date columns are coerced to the model's numeric encoding and factor
  levels are matched. Affects the Intermediate Output and RCA exports.
* `fit_earth()` drops a non-zero `newvar.penalty` (with a message) when
  case weights are present, since earth does not support that combination
  — previously it errored mid-fit (notably in market/appraisal mode where
  a skipped row is encoded as a zero weight).
* Character date columns are parsed with the same multi-format set as
  `validate_types()`, so any column that validates as a date also coerces
  (no more silent failures on `MM/DD/YYYY` and similar).

## Project model (regProj)

* New first-class **project** concept replaces the old per-file workflow.
  A project is a stable, named entity at a fixed location in the regProj
  tree (`<root>/<purpose>/<flat_segment>/<os>_in/...` and
  `<os>_out_<method>/...`), holding all input files, outputs, and
  settings together. Files within a project travel as a unit.
* Top-of-sidebar Project picker with sort (Recent / A-Z) and a New
  Project wizard that captures purpose, country, and admin-level
  cascade with country-specific terminology.
* Path layout is **flat**: `<purpose>/<country>_<state>_<county>_<city>_<project>/<os>_in/<file>`.
  All admin levels concatenated into one segment via `_`. Reduces depth
  from 13 to 8 directories under the regProj root. Encoded by
  `regproj_flat_segment()`, decoded by `regproj_parse_flat()`.
* Cross-OS multi-leaf scaffold: every project gets `mac_in`, `mac_out_*`,
  `ubuntu_in`, `ubuntu_out_*`, `win11_in`, `win11_out_*` siblings.
* Active-project model drives output paths automatically. The legacy
  Output Folder field and Project Location cascade are gone — output
  folder is set from the active project at session time.
* Section "Import Data" reads from the active project's `<os>_in/`;
  files added manually are picked up via Refresh.

## Geo data + variable-depth admin schema

* New `geo.sqlite` (in regProj root) replaces the old
  `.regproj-index.json`. Two tables: `countries` and `admin_entries`
  with variable depth via `(country, level, parent_codes)` index.
  Supports any number of admin levels per country.
* Comprehensive shipped data:
  * **US**: 51 states, 3,076 counties, ~21,000 incorporated places (Census).
  * **GB / DE / IT / FR / SE**: full GeoNames-derived state / county /
    city data (~45,000 entries combined).
  * **SG**: 1-level (planning area) data for Singapore.
* Total ~70,000 admin entries shipped; users can add more via the
  cascade's "create new" path. `regproj_index_get/put` API unchanged
  for callers, just SQLite-backed now.
* DE specifically uses `admin3_code` (Kreis) instead of GeoNames'
  `admin2_code` (Regierungsbezirk), matching how German appraisers
  think about administrative geography.

## Per-project settings (`projects.sqlite`)

* New `projects.sqlite` in regProj root replaces the user-level
  `settings.sqlite` for project model settings. Keyed by
  `(flat_segment, file_basename)`, separate JSON columns for earth,
  glmnet, and mgcv settings/variables/interactions.
* Public API: `get_project_settings()` and `set_project_settings()` —
  read/write project settings programmatically without launching the
  Shiny UI. Designed for ValEngr integration and batch automation.
* Settings now travel with the regProj tree (sync, backup, share with
  colleagues — all settings come too). User-level `settings.sqlite`
  retained only for locale defaults.

## Quarto report split: Generate + Convert

* **Section 9 — "Generate Quarto Report"**: writes a self-contained
  `.qmd` bundle (source + plot assets + reference.docx) under
  `<project>/mac_out_earth/<base>_qmd/`. No Quarto rendering happens
  at this step.
* **Section 10 — "Convert Quarto Report"**: file picker + format
  checkboxes (HTML / Word / PDF). Renders any `.qmd` file (not just
  earthUI-generated) to selected formats. Useful for combining
  reports across projects via Quarto's `{{< include >}}`.
* New exports: `generate_quarto_report()`, `convert_quarto_file()`.
* `report_data.rds` is now a lean ~hundreds-of-KB asset (down from
  hundreds of MB) — the bulky earth model object is no longer saved
  since the qmd reads pre-generated plots from disk.
* Quarto `.qmd` template now self-renders against a sibling
  `report_data.rds` when no `params` are passed, so
  `quarto preview Appraisal_1.qmd` works standalone.

## Settings tab

* New "regProj Root Folder" config in the Settings dropdown — set the
  location of the regProj tree per machine. Persisted in the per-OS
  prefs file at `R_user_dir("earthUI", "config")/prefs.json`.
* Resolution order: `REGPROJ_ROOT` env var → prefs file →
  per-OS default (`~/regProj` Mac/Linux, `C:/regProj` Win).

## Other

* Server startup extends `PATH` with `/usr/local/bin`, `/opt/homebrew/bin`,
  and TinyTeX bin paths so Quarto / pandoc / latex are findable from
  the Shiny session even when launched from a stripped-PATH context.
* Per-session DB connections (one geo, one projects) — eliminates
  per-render connection overhead.
* Sales Comparison Grid now exported as `build_sales_grid()` from
  the package (extracted from the old inst/app/sales_grid.R).
  Available to batch scripts and ValEngr.

# earthUI 0.7.0

## Per-Purpose Settings

* Switching purpose (General / Appraisal / Market) now fully clears all
  state: imported data, model results, tabs, variable configuration, and
  earth parameters. The app resets to a clean default state.
* Settings are now stored per purpose. localStorage keys include a purpose
  suffix (`_general`, `_appraisal`, `_market`), so each purpose maintains
  independent variable configuration, earth parameters, and interaction
  matrices for each imported file.
* SQLite backup keys encode purpose (`filename||purpose`), so settings
  persist across browser sessions per purpose+file combination.
* When a file is imported after a purpose switch, previously saved settings
  for that file+purpose combination are automatically restored.
* Backward-compatible migration detects old-format localStorage keys
  (without purpose suffix) and copies them to the correct purpose-suffixed
  key on first load.

## Fitting Modal Auto-Dismiss

* The fitting modal and backdrop now auto-dismiss 1.5 seconds after tabs
  finish rendering, preventing the overlay from blocking sidebar sections.
* On error, the backdrop is removed immediately while the modal stays
  visible with the close button so the user can read the error.

# earthUI 0.6.0

## Nord Theme Overhaul

* Navbar, sidebar, and all UI elements now use Bootstrap CSS variables
  (`var(--bs-body-bg)`, `var(--bs-body-color)`, `var(--bs-border-color)`)
  instead of hardcoded colors, so light and dark modes adapt correctly.
* Theme toggle button redesigned as a circular icon matching the glmnetUI
  and mgcvUI style.
* Custom `data-eui-theme` attribute replaces `data-bs-theme` to avoid
  conflicts with Bootstrap 5's built-in color mode system.
* Consistent Nord palette usage across earthUI, glmnetUI, and mgcvUI.

## Predictor Settings Table

* Column headers "Include", "Factor", and "Linear" now render vertically
  (rotated text), saving horizontal space.
* Checkbox columns grouped together (Include, Factor, Linear) with the
  Special dropdown moved after them.
* Header row uses `position: sticky` with a solid theme-aware background
  so it stays fixed while scrolling the variable list.
* Shows "Select a target variable above to configure predictors." when no
  target is selected, instead of an empty bordered bar.

## Data Tab Fix

* Appraisal mode data preview now shows all columns on initial load when
  no target or predictors are selected yet. Previously only the "row"
  column was visible.

## Section Info Icons

* Blue "?" info icons added to section headers (Sections 1-4, 6-9)
  with Bootstrap popovers describing each section's purpose.
* Info icons are right-justified within the sidebar using absolute
  positioning. Popovers open to the left to avoid overflow.

## Purpose Mode Improvements

* Switching purpose (General / Appraisal / Market) now clears all model
  results, RCA state, and report assets to prevent stale data.
* Purpose radio selection saved back to localStorage for persistence.

## Button Styling

* Buttons updated to use Frost-palette classes (`btn-primary`,
  `btn-outline-secondary`) instead of Aurora classes (`btn-success`,
  `btn-danger`) per the shared UI conventions.

# earthUI 0.2.0

## 3D Interaction Plot Fix

* Fixed plotly surface transposition bug — `add_surface()` uses `z[i,j]`
  at `(x[j], y[i])`, the transpose of base R `persp()`. Data point dots
  now correctly align with the surface in interaction contribution graphs.

## Report Rendering

* New `prepare_report_assets()` function pre-generates all plots (PNG + PDF)
  and pre-computes all data, so `render_report()` only runs Quarto/pandoc
  format conversion.
* `render_report()` gains `assets_dir` parameter to accept pre-generated
  assets for faster multi-format rendering.
* Report rendering now runs asynchronously via `callr::r_bg()` with a modal
  dialog showing elapsed time and Quarto progress. The app stays responsive.
* HTML reports now use KaTeX instead of MathJax for faster math rendering.
* Word reports now include "Page X of Y" page numbers in the footer.

## Post-Fit UX

* Results tabs appear immediately with "Waiting for processing to complete."
  messages while the model is fitting. The RCA Adjustments tab shows
  "7. Calculate RCA Adjustments & Download must first be initiated and
  completed."
* The Data tab persists across fits — same output IDs used before and after
  fitting, so the data table is never destroyed and recreated.
* Tab content renders on-demand (only the active tab computes).
* `auto_export_for_mgcv_()` (saveRDS) now runs in a background process to
  avoid blocking the UI after fit completion.
* Fit log and mgcv auto-export deferred via `session$onFlushed()` and
  `callr::r_bg()` so tabs appear instantly.

## Event Logging

* New session event log: `<filename>_earthui_log.txt` in the output folder
  records start/end timestamps and elapsed times for Model Fit, Download
  Output, RCA Adjustments, Sales Grid generation, and Report rendering.
  One file per data file, appended to across operations.

## Testing

* 71 new tests (1026 total):
  - `eval_g_function_()` direct unit tests for all component types (17 tests)
  - Plotly surface transposition verification (15 tests)
  - `prepare_report_assets()` and `render_report()` with `assets_dir` (15 tests)
  - Expanded export report error handling tests

# earthUI 0.1.3

## Locale & Regional Settings

* Country-based locale system with 31 country presets controlling CSV
  separator, decimal mark, thousands separator, date format ordering, and
  paper size.
* Country dropdown with individual override dropdowns for Paper, CSV sep,
  Decimal, and Date format.
* Global locale defaults saved via "Save as my default" (SQLite persistence),
  independent of per-file settings.
* Number formatting on plot axes and slope labels adapts to locale (e.g.,
  German uses period thousands, Finnish uses space, Swiss uses apostrophe).
* CSV import uses locale-aware separator and decimal mark.
* PDF reports respect locale paper size (Letter or A4).
* Date parsing tries locale-preferred format order first (MDY for US, DMY
  for Europe, YMD for Sweden/Japan).
* Currency symbols removed from all outputs --- earthUI is now
  currency-agnostic.

# earthUI 0.1.2

## Sales Comparison Grid (Appraisal Mode)

* New Sales Comparison Grid output for appraisal workflows — generates
  multi-sheet Excel workbooks with subject and up to 30 comps (3 per sheet).
* Auto-recommended comp selection based on gross adjustment percentage
  (< 25%), sorted by gross adjustment.
* Sale Price / Concessions / Net SP row with formula-based Net SP.
* Grouped rows for Location (longitude, latitude, area), Site Size
  (lot_size, site_dimensions), and Age (actual_age, effective_age) with
  combined value contributions.
* CQA|Residual row with formula-based remaining residual that auto-decreases
  as the appraiser fills in residual feature breakdowns.
* Adjusted Sale Price formula: Net SP + all adjustments above Total VC row.
* Sheet protection with locked formulas and data cells; unlocked residual
  feature input cells for appraiser entry.
* Haversine-based subject proximity calculation for each comp.

## Special Column Types

* Added new special column types: `dom`, `concessions`, `actual_age`,
  `effective_age`, `lot_size`, `site_dimensions`, `area`.
* Special type badges displayed next to variable names in predictor settings.
* Single-assignment enforcement: only one column per special type (except
  `display_only` which allows multiple).

## Intermediate Output Improvements

* Ranking columns (`residual_sf`, `cqa_sf`, `residual`, `cqa`) moved to
  leftmost position in the Step 6 download for easier CQA evaluation.
* Excel number formatting: `residual_sf` as currency (2 dp), `residual` as
  currency (0 dp), `cqa_sf` and `cqa` as numeric (2 dp).

## UI Improvements

* Fit Earth Model button always visible (no longer inside collapsible section).
* Minimum sidebar width (500px) to prevent predictor settings column overlap.
* Wider notification popups (450px) with word-wrapping for long file paths.

## Bug Fixes

* Cross-validated R-squared now uses `model$cv.rsq` directly instead of
  parsing `cv.rsq.tab`.

# earthUI 0.1.0

## Core

* Shiny GUI for interactive earth (MARS) model building.
* Data import from CSV and Excel files with automatic snake_case column naming.
* Variable configuration with include, factor, and linear-only controls.
* Full exposure of all `earth()` parameters through the GUI with inline help
  popovers.
* Cross-validation automatically enabled for interaction and variance models.
* Settings persistence via `localStorage` across browser sessions.
* Light/dark mode toggle.

## Allowed Interactions

* Allowed interaction matrix for constraining degree >= 2 models.
* CSS sticky headers — top row and left column stay visible while scrolling.
* Click variable name to toggle all its interactions (symmetric toggle).
* Allow All / Clear All buttons.

## Multivariate Models

* Multi-response model support: select multiple target variables to fit
  `cbind()` earth models.
* Per-response equation display, summary metrics, and ANOVA decomposition.
* All plot functions accept `response_idx` to select which response to
  visualize.
* Observation weights via the `weights` parameter in `fit_earth()`.

## Model Fitting

* Asynchronous background fitting via `callr::r_bg()` with real-time trace
  display and elapsed timer.
* Automatic `nfold`/`ncross` adjustment for variance models.
* Synchronous fallback when callr is unavailable.

## Visualization

* Variable importance bar charts.
* Per-variable contribution plots with piecewise-linear slope labels and knot
  markers.
* g-function plots: 1D line, interactive 3D surface (plotly), static 3D
  perspective (`persp()`), and filled contour.
* Correlation matrix heatmap.
* Diagnostic plots: residuals vs fitted, Q-Q, actual vs predicted.
* Adaptive axis formatting — 3 decimal places for lat/long-scale data,
  commas for large values.
* Slope labels scaled to axis range (`/0.001` for small-range, `/unit` for
  large-range variables).

## Model Equation

* LaTeX equation with g-function notation rendered via MathJax in the app
  and native LaTeX in PDF reports.
* Full LaTeX special character escaping (`_`, `$`, `%`, `&`, `#`).

## Report Export

* Publication-quality HTML, PDF, and Word reports via Quarto.
* Both 3D perspective and contour plots for degree-2 groups in static reports.
* Earth model summary, ANOVA decomposition, and raw earth output sections.
