Today was very productive. Lots of changes introduced. I switched to Claude Sonnet 4.5. It really was very helpfull, more than Gemini.

Done

• The monorepo a proper structure now. I’m using UV Workspaces to work with packages.
• Building: cloudbuild.yml. That file will be added alongside the Dockerfile on each standalone job/service. The deployment gets done from the root.
• Makefile was also introduced. It’s able to build and update my project infrastructure, running cloud build and terraform.
• Tests: they are run with pytest from the root directory. Modules can be tested by themself or everything together.
• AV Extractor: started to update this job to extract and load into GCS.
• AV Loader: draft. Reads from GCS and writes into BQ.
• Airflow: draft of the DAG to extract and load.

Next steps:

• Work on the AV Extractor.
  • It needs to read from BQ the list of Symbols to get (market and other data).
  • Query them and write into GCS.
  • Both for backfill and daily.
  • Review the BQ schemas, from the tables we read and the tables we write to.
  • Work in the data loader.