Web scraping trends by Oxylabs reveal the most popular categories for Black Friday this year.
Black Friday means online retailers are increasingly putting more effort into monitoring the pricing of their competitors before deciding on a discount themselves. New data from Oxylabs, a leading web data gathering solutions provider, reveals the product categories that could drive the most competition this year.
Web scraping is a technology which allows large-scale collecting of public web data, which has become mainstream among ecommerce companies in the last few years. Retailers collect data on the pricing of their competitors, product availability, shipping information, and promotions, to name a few. This information allows retailers to make informed decisions on discount strategies, pertinent to competitive sales periods including Black Friday.
Oxylabs analysed and compared two periods – the lead up to Black Friday 2021 (1-5 November 2021), and the actual days of sales last year (26-30 November 2020). Based on this data, the most scraped product categories in the world’s largest marketplaces at the beginning of November 2021 were:
- Clothing and shoes
- Books
- Home and kitchen
- Grocery & Gourmet Food
- Automotive
The Oxylabs web scraping infrastructure traffic rates show that online retailers were already three times more active during the beginning of November 2021 – compared to the five days of Black Friday in 2020.
Going deeper into the product list, the most researched products by the retailers at the beginning of November were posters and prints (1.4% of all web scraping jobs in said period), rugs (1.2%), women’s jeans (1%), men’s jeans (0.9%), and health-themed books (0.67%). Over 25,000 different product subcategories were scraped.
According to Gediminas Rickevičius, Director of Strategic Partnerships at Oxylabs: “This data reveals to consumers which categories will be the most competitive, allowing sellers to then offer the best price rates.
Consumers can expect the most discounts in these product categories, as this is where sellers are putting the most effort into following their competition. Sellers analyse the whole competitive landscape for a particular product and formulate their offer accordingly.
With global supply chain issues and forecasted shipping delays threatening the largest sales festival of the year, retailers are looking for smarter ways to get a competitive edge and leveraging publicly available data for it.”
Clothing dominated the full list of web scraping jobs and the whole apparel category was getting more traffic than last year, potentially influenced by the pandemic shopping habits. With lockdowns still in place in many parts of the world during last year’s Black Friday, many people abstained from updating their closets and focused on other items in their shopping list. This year, however, retailers are strongly betting on the clothing category to drive the most interest from consumers and are preparing for it.
Posters and prints were the most popular items in Black Friday 2020 with 3% of all unique web scraping jobs, followed by history books (1.5%), men’s t-shirts (1%), humanities books (1%), and health-themed books (0.9%).
During the 2020 Black Friday sales, the top five most scraped product categories were:
- Books
- Clothing, Shoes & Jewellery
- Home & Kitchen
- Automotive
- Sports & Outdoors
Grocery and gourmet food is a new rising product category this year, revealing a trend that food retailers are increasingly putting more emphasis on online sales.
Web scraping is becoming a crucial technology for ecommerce businesses. Oxylabs research in the UK shows that 57% of the UK’s online stores use the technology to get essential insights into the competitive landscape.