Automated Extraction of Clean News URLs from Google News RSS

somdn_product_page

This workflow automates the process of extracting and decoding URLs from Google News RSS feeds to obtain clean, direct article links. It begins with a manual trigger for test runs and fetches the latest news RSS feed from Google News in a specified language and region. The workflow limits the number of articles to avoid excessive requests, then parses the RSS data to extract the encoded URLs.

Next, it sends requests to Google’s URL decoding service, reverse-engineering the obfuscated news URLs by dynamically constructing hidden decoding variables such as signature and timestamp from HTML content. These variables are used in a POST request to Google’s decoding endpoint to retrieve the actual article URLs. The workflow concludes with a data aggregation step, compiling all decoded URLs into a single structured object.

This process is useful for aggregators, researchers, or developers who need reliable, direct access to news articles from Google News RSS feeds—especially when URLs are encoded or obfuscated, and manual decoding is impractical. It’s particularly valuable in scenarios that require automated, scalable news link harvesting with minimal manual intervention.

Node Count

11 – 20 Nodes

Nodes Used

aggregate, code, html, httpRequest, limit, manualTrigger, rssFeedRead, set, stickyNote

Reviews

There are no reviews yet.

Be the first to review “Automated Extraction of Clean News URLs from Google News RSS”

Your email address will not be published. Required fields are marked *