Ingest comdirect #5
@@ -0,0 +1,107 @@
|
||||
# finanz-cockpit Agent Instructions
|
||||
|
||||
## Setup Commands
|
||||
|
||||
```bash
|
||||
# Install dependencies
|
||||
uv sync
|
||||
|
||||
# Run migrations
|
||||
alembic upgrade head
|
||||
|
||||
# Run all tests
|
||||
pytest
|
||||
|
||||
# Run a single test file
|
||||
pytest tests/test_model_b4wexport.py -v
|
||||
```
|
||||
|
||||
## Project Structure
|
||||
|
||||
- **`src/`**: Application code
|
||||
- `main.py`: CLI entrypoint for CSV ingestion
|
||||
- `db.py`: Database connection management (uses global `_engine` pattern)
|
||||
- `models/`: Pydantic models + SQLAlchemy tables for b4w and comdirect imports
|
||||
- `ingest/`: Data ingestion logic per source
|
||||
- **`tests/`**: Tests with fixtures in `conftest.py`
|
||||
- **`alembic/versions/`**: Database migrations
|
||||
|
||||
## Key Patterns & Gotchas
|
||||
|
||||
### Database Configuration
|
||||
|
||||
```python
|
||||
from src.db import get_engine, get_session, set_engine
|
||||
|
||||
# Development: use sqlite://
|
||||
os.environ["DATABASE_URL"] = "sqlite://"
|
||||
|
||||
# Production/default: sqlite:///finanz.db
|
||||
engine = get_engine() # reads DATABASE_URL or uses default
|
||||
with get_session() as session:
|
||||
session.query(...).all()
|
||||
```
|
||||
|
||||
**Critical**: Tests override engine via `conftest.py`. Never assume global state - always use `get_session()` context manager.
|
||||
|
||||
### CSV Format Expectations
|
||||
|
||||
- **b4w**: German bank export (CSV) from `/mnt/anbau/Software/Python/finanz-cockpit/scripts/dummy_data/b4w_export_dummy_01.csv`
|
||||
- **comdirect**: German bank export (CSV) with fields like `Buchungstag`, `Umsatz in EUR`, `Vorgang`
|
||||
|
||||
### Testing Workflow
|
||||
|
||||
```bash
|
||||
# Tests automatically run migrations via conftest fixture
|
||||
pytest tests/ -v --tb=short
|
||||
|
||||
# To test without migrations (in-memory SQLite), use DATABASE_URL="sqlite://"
|
||||
export DATABASE_URL="sqlite://" pytest tests/test_db.py
|
||||
```
|
||||
|
||||
### Environment Variables
|
||||
|
||||
- `DATABASE_URL`: Database connection string (default: `sqlite:///finanz.db`)
|
||||
|
||||
## Data Models
|
||||
|
||||
Two sources with shared patterns:
|
||||
|
||||
- **b4w**: `B4wExport`, `Category`, transactional import to `import_b4w` table + `category` table
|
||||
- **comdirect**: `ComdirectGiro`, `ComdirectCreditCard`, `ComdirectDepot`, import to `comdirect_giro` table
|
||||
|
||||
All models inherit from common base in `src/models/_common.py`:
|
||||
- `_hash()`: SHA256-based deduplication key
|
||||
- `_parse_date_german()`: Handles "07.09.2026" format and values like "offen", "--"
|
||||
- `_parse_decimal_german()`: Converts "1,23" to Decimal
|
||||
|
||||
## Factory Pattern for Testing
|
||||
|
||||
Import from `tests/factories.py`:
|
||||
|
||||
```python
|
||||
from tests.factories import B4wExportFactory, ComdirectGiroFactory
|
||||
from src.constants import Currency
|
||||
|
||||
B4wExportFactory.__faker__ = Faker("de_DE")
|
||||
ComdirectGiroFactory.__faker__ = Faker("de_DE")
|
||||
|
||||
# Use in tests:
|
||||
item = B4wExportFactory.build( # build() for partial, create() for full
|
||||
Amt="50.00",
|
||||
RmtInf="Supermarket",
|
||||
)
|
||||
```
|
||||
|
||||
## Common Mistakes to Avoid
|
||||
|
||||
1. **Assuming global engine state**: Always use `get_session()` context manager
|
||||
2. **Ignoring German date formats**: Use `_parse_date_german()` for CSV parsing
|
||||
3. **Forgetting migration setup in tests**: Tests auto-migrate, but integration tests with real db need explicit upgrade
|
||||
4. **Wrong Faker locale**: Use `Faker("de_DE")` for realistic names/IBANs
|
||||
|
||||
## Testing Tips
|
||||
|
||||
- Run single test: `pytest tests/test_model_b4wexport.py -v`
|
||||
- View migrations: `ls alembic/versions/`
|
||||
- Test data location: `/scripts/dummy_data/`
|
||||
Reference in New Issue
Block a user