E-commerce Price Tracker is a production-ready price tracking and scraping platform. Input store URLs, execute automated scheduled scrapes, and extract structured product name, price, and stock availability — all rendered in a premium glassmorphic dark dashboard with dynamic grid card views and direct Excel/CSV downloads.
- Overview
- Application Preview
- Features
- Architecture
- Tech Stack
- Project Structure
- Installation
- Usage
- API Reference
- Configuration
- Design Decisions
- License
E-commerce Price Tracker provides a local and cloud-compatible workflow for product price intelligence. The backend launches headless Playwright instances via Crawlee, scrolls to execute dynamic content loading, parses multiple structured data layouts (JSON-LD schemas, OpenGraph, or CSS Selectors), and serves the real-time dataset to a dark glassmorphic client-side dashboard.
Users can:
- Paste single/multiple product and listing URLs to crawl stores.
- Toggle advanced scraping features (Apify rotating proxies, infinite scrolls, and category product enqueuing).
- Monitor progress live and view real-time metrics (Total found, in-stock count, average market price).
- Switch between interactive layout view presentation modes (Table List vs. Card Grid).
- Download dataset exports on-demand in
.json,.csv, or formatted Excel.xlsxspreadsheets.
| Feature | Description |
|---|---|
| 🛍️ Multi-Source Scraping | Crawls e-commerce websites utilizing headless browser execution via Playwright |
| 🔄 Crawling & Enqueuing | Discovers and crawls product pages inside collection listings automatically |
| ⏱️ Infinite Scroll Support | Simulates scrolls to capture lazy-loaded dynamic image assets and layout content |
| 🛡️ Proxy Rotation | Integrated proxy rotation support to prevent IP blocking and rate limits |
| 📊 Dynamic Metrics | Live panel computing average catalog prices, in-stock counts, and total products |
| 🎛️ Dual View Layout | Toggle layout view presentation between a tabular data list and interactive cards |
| 💾 Scrape Run History | Persists execution results to disk so users can review past datasets |
| 🧹 New Search Reset | Reset configuration forms, dashboard counters, and database status immediately |
| 📅 Cron Automations | Node-cron scheduling frequencies (Hourly, Daily, Weekly, or Custom Cron patterns) |
| 📥 Multi-Format Exports | Real-time backend sheet exporter producing clean CSV, JSON, and Microsoft Excel sheets |
| 🚀 Dual Run Target | Runs locally as a standalone Web Service or on the Apify Cloud Platform as a Batch Actor |
┌───────────────────────────────────────────────────────────────────┐
│ Glassmorphism Dashboard │
│ │
│ UrlInputs ──► SegmentedTabs ──► ViewToggles (Table / Cards Grid) │
│ │ │ │ │
│ └────── POST /api/scrape ◄──────────┘ │
│ GET /api/status │
└──────────────────────────────┬────────────────────────────────────┘
│
▼
┌───────────────────────────────────────────────────────────────────┐
│ Express Backend │
│ │
│ Middleware: CORS + Static Assets Router + JSON Parser │
│ │
│ /api/scrape ─► scraper.js ───────► PlaywrightCrawler (Crawlee) │
│ ├► JSON-LD Schema Extractor │
│ ├► Meta OG/Twitter Fallbacks │
│ └► CSS Selector Fallbacks │
│ │
│ /api/config ─► storage/config.json (Persistent configuration) │
│ /api/export ─► xlsx-exporter / csv-generator / json-stream │
│ │
│ node-cron triggers scheduled background crawls on active URLs │
└───────────────────────────────────────────────────────────────────┘
| Layer | Technology |
|---|---|
| Backend | Node.js, Express, Crawlee, Playwright |
| Provider | Apify SDK & CLI compatibility layers |
| Parser Engine | JSON-LD schema search, OpenGraph, custom CSS fallback matchers |
| Scheduler | Node-cron |
| Persistence | File-based local store (storage/config.json, storage/datasets/) |
| Frontend | Vanilla HTML5/CSS3 (Inter Font, Volcanic HSL Glassmorphism, animations) |
| Sheet Engine | xlsx node package, CSV string stream generators |
ecommerce-price-tracker/
│
├── .actor/
│ └── input_schema.json # Apify Actor Console Inputs configuration
│
├── public/
│ ├── index.html # Main dashboard structural layouts
│ ├── style.css # Volcanic theme visual styles & animations
│ └── app.js # Dashboard state, metrics, and search logic
│
├── src/
│ ├── scraper.js # Crawlee/Playwright engine with scroll & parsing
│ └── server.js # Express API Router, exports, and cron scheduler
│
├── screenshots/
│ ├── 01-dashboard.png # Dashboard configuration view screenshot
│ └── 02-tracked-products.png # Tracked product results grid view screenshot
│
├── storage/ # Local database stores (gitignored)
│ ├── config.json # Local persistent configurations
│ ├── history/ # Persistent scraping history records
│ └── datasets/ # Scraped items dataset runs
│
├── apify.json # Apify Platform integration configuration
├── package.json # NPM script pathways and dependency versions
├── .gitignore # Git track rules
└── README.md # Service setup and schema guide
git clone https://github.com/crastatelvin/ecommerce-price-tracker.git
cd ecommerce-price-trackerInstall project dependencies and boot the local Express server:
npm install
npm startDashboard Access: http://localhost:3000
- Open the local dashboard page in your web browser.
- Enter target store homepage or product URLs into the configuration box.
- Configure your automated tracking schedule using the Segmented Tab Bar.
- Click Run Tracker Now to trigger the scraper.
- Filter, search, and analyze catalog details in the results grid.
- Export datasets using the download panels.
| Method | Endpoint | Description |
|---|---|---|
GET |
/api/config |
Load persistent scraper configuration |
POST |
/api/config |
Save scheduling and advanced options configurations |
POST |
/api/scrape |
Trigger manual scraping run immediately |
GET |
/api/status |
Fetch running job state and scraped dataset results |
POST |
/api/clear |
Reset active job state to idle |
GET |
/api/history |
Get list of historical run metadata |
GET |
/api/history/:id |
Retrieve detailed scraped products from a past run |
GET |
/api/export/json |
Export results dataset as JSON |
GET |
/api/export/csv |
Export results dataset as CSV |
GET |
/api/export/xlsx |
Export results dataset as Microsoft Excel Spreadsheet |
Scraper inputs are stored locally in storage/config.json. When deployed to Apify, configurations map directly to the Actor Console Input schema:
{
"urls": ["https://zivanche.com"],
"schedule": "daily",
"customCron": "0 9 * * *",
"useProxy": false,
"crawlLinks": true,
"maxScrolls": 1
}- Playwright Crawler & Headless Mode: Crawlee with Playwright ensures full execution of JS components to scrape dynamic client-rendered products.
- Three-Tier Product Data Parser: Structured JSON-LD schema parsing is used first. If absent, fallback extraction matches Meta OG tags or CSS selectors to maximize data capture.
- Deduplicated Category Traversal: Category crawling enqueues sub-links into Crawlee's Request Queue, which handles de-duplication automatically to prevent scrapers from visiting identical pages.
- Local SQLite-free File Persistence: Using JSON files inside
/storageprovides a lightweight data layer without the overhead of setting up a SQL database.
This project is licensed under the MIT License. See LICENSE if present.
Built by Telvin Crasta · Production-ready · Live today
⭐ If this helped you track e-commerce prices faster, star the repo.