Skip to content

Update Data Scraper for Share Food Program #727

@marcbachan

Description

@marcbachan

Is your feature request related to a problem? Please describe.
Related to #697. We need to update the Share Food Program script to fetch resources from:

https://www.sharefoodprogram.org/wp-json/wpgmza/v1/features/base64eJyrVkrLzClJLVKyUqqOUcpNLIjPTIlRsopRMo5R0gEJFGeUFni6FAPFomOBAsmlxSX5uW6ZqTkpELFapVoABaMWvA

The existing Python script should work for the raw data, but it currently wants to write to Firebase, and not Supabase. We also have a Lambda function in AWS to automate it, but per @gcardonag this has been failing continually. The biggest hurdle here is to make sure it writes to Supabase.

Describe the solution you'd like
The current script is written in Python. As discussed with @icycoldveins the best solution may be to write this up as a TypeScript edge function that we can manage through the Supabase web UI.

Describe alternatives you've considered
Running Python scripts on our local machines and automating them with Lambda/Github Actions can be finnicky and tough to maintain. Either we streamline the Python scripts or do this as an Edge function.

Metadata

Metadata

Assignees

Labels

Data CircleTickets related to the Data Circle

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions