Develop an ingestion and storage system #4

Closed
opened 2026-01-13 11:33:03 -05:00 by preteck · 0 comments
Owner

Develop an ingestion system that:

  • Endpoint that extracts data with specified date
    • If no date, the latest data.
  • Data storage
  • Integration Layer
    • Checks for duplicates.
    • Checks if previous data is missing.
    • Checks for future data between now and the latest data.
    • Runs every 65 seconds to update the data with latest data.
Develop an ingestion system that: - [ ] Endpoint that extracts data with specified date - [ ] If no date, the latest data. - [ ] Data storage - [ ] Integration Layer - [ ] Checks for duplicates. - [ ] Checks if previous data is missing. - [ ] Checks for future data between now and the latest data. - [ ] Runs every 65 seconds to update the data with latest data.
Sign in to join this conversation.
No Label
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: preteck/hanchuess-solar#4