DiscountBot
DiscountBot este robotul Discount.ro. Citește pagini publice de produs din magazine online ca să afișeze prețul și stocul lor în comparația de prețuri. Regulile lui sunt mai jos, în engleză, pentru administratorii de site-uri din orice țară.
Purpose and user agent
DiscountBot is the crawler of Discount.ro, a Romanian price comparison and coupon site operated by Tidy Corp SRL (Calea Dudești nr. 94, București, România). It reads public product pages of online shops so that Discount.ro can show each shop's current price and availability next to the shop's name, with a link to the shop.
- User agent
Mozilla/5.0 (compatible; DiscountBot/1.0; +https://discount.ro/despre-noi/discountbot/)- Identification
- Every request is signed with HTTP Message Signatures (Web Bot Auth, Ed25519). Public keys: https://discount.ro/.well-known/http-message-signatures-directory
- Signature-Agent
"https://discount.ro"- Not DiscountBot
- A request that carries this name without a valid signature from the keys above was not sent by us.
Pages and data fetched
- Pages: single product pages of online shops, at addresses the shop itself published. DiscountBot does not follow links and does not discover pages by crawling a site.
- Method:
GETonly, the HTML document only. No images, scripts, style sheets or media are requested. - Never requested: search results, carts, checkouts, account pages, or anything behind a login.
- Data read from a page: the product's price and currency, availability, and title, from the page's structured data (JSON-LD, microdata).
Schedule and request rate
- At most one request every 5 seconds to one site, over one connection at a time; never in parallel to the same site.
- The same page is requested at most once every 12 hours.
- It runs around the clock in small batches, so there is no daily peak.
- A
Crawl-delayin robots.txt longer than 5 seconds is followed (up to 60 seconds; a longer one means the site is not read).
Data retention and reuse
- Kept: the price, availability and title read, and the time of the read. Price changes are kept for 90 days to draw the product's price history.
- Not kept: the page's HTML is discarded once those values are read.
- Use: the price comparison on Discount.ro, and price statistics and reports made by the operator, Tidy Corp SRL.
- Not used to train AI models.
robots.txt handling
- A site's
/robots.txtis read before its first page and again after 24 hours. - DiscountBot follows the group for
User-agent: DiscountBot, or the*group when there is none. A disallowed page is not requested. - When robots.txt cannot be read (a server error or no answer), the site is not read until it can.
- To opt out:
User-agent: DiscountBotDisallow: /
Refusals, rate limits, challenges, logins, forms and transactions
- Refusals and rate limits: a
401,403,429or503answer is taken as "not now". The request is not repeated and the whole site is left alone for 24 hours, or longer whenRetry-Aftersays so. - Challenges: DiscountBot never solves or works around a CAPTCHA, a JavaScript challenge or any other check. A challenge page is treated as a refusal.
- Logins: it has no accounts and never signs in.
- Forms: it never submits a form and sends no
POSTrequest. - Transactions: it never adds to a cart, reserves or buys anything.
Contact
To report a problem with DiscountBot, to ask for a different rate, or to be left out: office@discount.ro or the contact page. Please include your domain and, if you can, the time and address of a request.