Ingest comdirect #5
@@ -0,0 +1,107 @@
|
|||||||
|
# finanz-cockpit Agent Instructions
|
||||||
|
|
||||||
|
## Setup Commands
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Install dependencies
|
||||||
|
uv sync
|
||||||
|
|
||||||
|
# Run migrations
|
||||||
|
alembic upgrade head
|
||||||
|
|
||||||
|
# Run all tests
|
||||||
|
pytest
|
||||||
|
|
||||||
|
# Run a single test file
|
||||||
|
pytest tests/test_model_b4wexport.py -v
|
||||||
|
```
|
||||||
|
|
||||||
|
## Project Structure
|
||||||
|
|
||||||
|
- **`src/`**: Application code
|
||||||
|
- `main.py`: CLI entrypoint for CSV ingestion
|
||||||
|
- `db.py`: Database connection management (uses global `_engine` pattern)
|
||||||
|
- `models/`: Pydantic models + SQLAlchemy tables for b4w and comdirect imports
|
||||||
|
- `ingest/`: Data ingestion logic per source
|
||||||
|
- **`tests/`**: Tests with fixtures in `conftest.py`
|
||||||
|
- **`alembic/versions/`**: Database migrations
|
||||||
|
|
||||||
|
## Key Patterns & Gotchas
|
||||||
|
|
||||||
|
### Database Configuration
|
||||||
|
|
||||||
|
```python
|
||||||
|
from src.db import get_engine, get_session, set_engine
|
||||||
|
|
||||||
|
# Development: use sqlite://
|
||||||
|
os.environ["DATABASE_URL"] = "sqlite://"
|
||||||
|
|
||||||
|
# Production/default: sqlite:///finanz.db
|
||||||
|
engine = get_engine() # reads DATABASE_URL or uses default
|
||||||
|
with get_session() as session:
|
||||||
|
session.query(...).all()
|
||||||
|
```
|
||||||
|
|
||||||
|
**Critical**: Tests override engine via `conftest.py`. Never assume global state - always use `get_session()` context manager.
|
||||||
|
|
||||||
|
### CSV Format Expectations
|
||||||
|
|
||||||
|
- **b4w**: German bank export (CSV) from `/mnt/anbau/Software/Python/finanz-cockpit/scripts/dummy_data/b4w_export_dummy_01.csv`
|
||||||
|
- **comdirect**: German bank export (CSV) with fields like `Buchungstag`, `Umsatz in EUR`, `Vorgang`
|
||||||
|
|
||||||
|
### Testing Workflow
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Tests automatically run migrations via conftest fixture
|
||||||
|
pytest tests/ -v --tb=short
|
||||||
|
|
||||||
|
# To test without migrations (in-memory SQLite), use DATABASE_URL="sqlite://"
|
||||||
|
export DATABASE_URL="sqlite://" pytest tests/test_db.py
|
||||||
|
```
|
||||||
|
|
||||||
|
### Environment Variables
|
||||||
|
|
||||||
|
- `DATABASE_URL`: Database connection string (default: `sqlite:///finanz.db`)
|
||||||
|
|
||||||
|
## Data Models
|
||||||
|
|
||||||
|
Two sources with shared patterns:
|
||||||
|
|
||||||
|
- **b4w**: `B4wExport`, `Category`, transactional import to `import_b4w` table + `category` table
|
||||||
|
- **comdirect**: `ComdirectGiro`, `ComdirectCreditCard`, `ComdirectDepot`, import to `comdirect_giro` table
|
||||||
|
|
||||||
|
All models inherit from common base in `src/models/_common.py`:
|
||||||
|
- `_hash()`: SHA256-based deduplication key
|
||||||
|
- `_parse_date_german()`: Handles "07.09.2026" format and values like "offen", "--"
|
||||||
|
- `_parse_decimal_german()`: Converts "1,23" to Decimal
|
||||||
|
|
||||||
|
## Factory Pattern for Testing
|
||||||
|
|
||||||
|
Import from `tests/factories.py`:
|
||||||
|
|
||||||
|
```python
|
||||||
|
from tests.factories import B4wExportFactory, ComdirectGiroFactory
|
||||||
|
from src.constants import Currency
|
||||||
|
|
||||||
|
B4wExportFactory.__faker__ = Faker("de_DE")
|
||||||
|
ComdirectGiroFactory.__faker__ = Faker("de_DE")
|
||||||
|
|
||||||
|
# Use in tests:
|
||||||
|
item = B4wExportFactory.build( # build() for partial, create() for full
|
||||||
|
Amt="50.00",
|
||||||
|
RmtInf="Supermarket",
|
||||||
|
)
|
||||||
|
```
|
||||||
|
|
||||||
|
## Common Mistakes to Avoid
|
||||||
|
|
||||||
|
1. **Assuming global engine state**: Always use `get_session()` context manager
|
||||||
|
2. **Ignoring German date formats**: Use `_parse_date_german()` for CSV parsing
|
||||||
|
3. **Forgetting migration setup in tests**: Tests auto-migrate, but integration tests with real db need explicit upgrade
|
||||||
|
4. **Wrong Faker locale**: Use `Faker("de_DE")` for realistic names/IBANs
|
||||||
|
|
||||||
|
## Testing Tips
|
||||||
|
|
||||||
|
- Run single test: `pytest tests/test_model_b4wexport.py -v`
|
||||||
|
- View migrations: `ls alembic/versions/`
|
||||||
|
- Test data location: `/scripts/dummy_data/`
|
||||||
Reference in New Issue
Block a user