Automated GitHub Trending Projects Data Collection to Supabase

somdn_product_page

This n8n workflow automates the process of scraping GitHub trending repositories based on different timeframes (daily, weekly, monthly) and stores the extracted data into a Supabase database.

The workflow starts with three schedule triggers that run at specified intervals for daily, weekly, and monthly updates. Each trigger sets a variable ‘type’ indicating the timeframe.

For each scheduled run, the workflow uses Firecrawl to scrape the GitHub trending page corresponding to the timeframe (determined by the ‘type’ variable). The scraping excludes non-essential content to focus on main project data.

The ‘AI Agent’ node then processes the scraped Markdown content, extracting key project details such as project name, URL, description, main programming language, and star count. It also tags each project with its respective ‘type’ (daily, weekly, monthly).

Finally, the workflow inserts this structured data into the ‘githubtrending’ table in Supabase, creating a comprehensive and up-to-date repository of popular GitHub projects for analysis or display purposes.

This setup is particularly useful for developers, data analysts, or teams who want to track trending repositories automatically for research, dashboards, or reporting.

Node Count

11 – 20 Nodes

Nodes Used

@mendable/n8n-nodes-firecrawl.firecrawl, @n8n/n8n-nodes-langchain.agent, @n8n/n8n-nodes-langchain.lmChatOpenAi, scheduleTrigger, set, stickyNote, supabaseTool

Reviews

There are no reviews yet.

Be the first to review “Automated GitHub Trending Projects Data Collection to Supabase”

Your email address will not be published. Required fields are marked *