RSS Reader Duplicate Story Merger

WHAT IT DOES

RSS Reader Duplicate Story Merger scans the current page for RSS feed items and identifies duplicate stories across different feeds by comparing titles, URLs, and content snippets. It extracts story titles, publication dates, feed sources, and content previews from RSS reader pages. The floating panel shows grouped duplicate stories with merge controls, allowing users to select a preferred version (newest, oldest, or manual selection). Merge options include consolidating feed sources, keeping the best content preview, and tracking merged stories. Duplicate count statistics, source overlap analysis, and one-click export of merged story data as JSON or text are all provided. Custom duplicate detection thresholds support title similarity percentage, URL normalization, and date window adjustments.

WHERE IT RUNS

Works on any RSS reader page, including Feedly, Inoreader, Feedbin, The Old Reader, and other web-based RSS readers. The script intelligently detects feed items using common RSS reader selectors and can also fall back to general article-like elements on any page with structured content lists. It works best on Feedly and Inoreader but can detect stories on most RSS reader platforms.

HOW TO USE

The panel displays statistics showing total stories, unique stories, duplicate groups, and duplicate counts. Click "Scan RSS Page" to detect all stories and identify duplicate groups. Each duplicate group shows the story title, number of copies, and source tags. Click "Merge (keep newest)" to merge a group into a single story, keeping the newest version. Use "Merge All Groups" to process all duplicate groups at once. Adjust the similarity threshold slider to control how aggressively duplicates are detected (lower = more duplicates detected). Choose comparison method (Title, Content, or Both) and merge preference (Keep Newest, Keep Oldest, Manual Selection). Export the report as text or JSON for documentation.

MECHANISM — TEXT SIMILARITY WITH JACCARD COEFFICIENT

The script uses the Jaccard similarity coefficient to compare stories. Each story's title (or content) is tokenized into word sets, and the similarity between two sets is calculated as the size of their intersection divided by the size of their union. This method is robust against word order differences and common variations in article titles. The similarity threshold (default 0.7) determines whether two stories are considered duplicates. For URL-based detection, the script normalizes URLs by removing trailing slashes and query parameters, then compares the normalized versions. Date extraction uses pattern matching to identify publication dates in various formats, with fallback to the current timestamp.

THE PANEL

The floating panel is fully draggable and remembers its position per domain. Click the minimize button to collapse it to a small bubble, or press Escape to hide it. The panel automatically adjusts to different screen sizes and can be resized to show more content. All settings, including similarity threshold, comparison method, and merge preference, are saved per domain. The panel integrates smoothly with any page layout using the BGPanel framework, maintaining clean separation from page content.

PLEASE NOTE

This script requires the free BotGentz extension to run. It does not send any data to external servers; all extracted story data, duplicates, and settings remain entirely local to your browser using GM_setValue. The script never reads password fields, hidden inputs, or file inputs. It stores only story titles, URLs, sources, content snippets, and dates from publicly visible RSS page content. The script merges duplicate stories in the local view only and does not modify the actual RSS feed data. Data persists across browser sessions and is isolated per browser profile.