Netshoes: Powering the Search for a Better E-commerce Experience

Netshoes is a multi-store platform that currently runs 32 domains, such as zattini.com.br and partner stores saopaulomania.com.br and shoptimao.com.br. Their mission is to inspire and transform people’s lives with sports and leisure by providing a great shopping experience through their innovative online platform. As they were experiencing growth at scale, Netshoes was looking for a platform that could support a high volume of queries, connecting directly with the Latin America regions. The Netshoes team, led by Diego Peroni, transitioned the e-commerce platform’s search solution to Elasticsearch. They chose Elastic as their new solution after discovering its ease of configurability—enabling them to quickly provide more relevant search results with deeper personalization while supporting their mission of enhanced customer satisfaction and site performance.

Netshoes previously used Endeca, the search function of Oracle ATG Web Commerce Search, to power search on their e-commerce platform. With Endeca, they relied on a third-party company to handle configurations and search functionality. However, it was difficult to match the right Netshoes employee to complete tasks with the third party, and fixes would sometimes take between 12 and 24 hours.

Netshoes decided to migrate away from Endeca and build their own platform to eliminate reliance on external vendors. They did so by starting from scratch with the Elastic Stack.

Netshoes had a hard deadline to get their Elastic-powered platform into production ahead of Black Friday, the largest retail shopping day in Brazil. Peroni and his team started by indexing the entire Netshoes catalog, which contained 89 million documents. They then successfully migrated the other 31 of 32 Netshoes e-commerce domains to the new Elastic Stack powered platform. IT staff noted a significant reduction in operational overhead and time in comparison to when they were using Endeca.

The next step was to migrate netshoes.com.br over the next three weeks. As their primary web domain, it experienced the highest web traffic (equivalent to the other 31 combined) and contained 15 million documents. During controlled load tests, they discovered that the clusters in their new platform could not handle the volume of search requests expected on Black Friday. Due to high site traffic and the lack of optimization, the Netshoes e-commerce platform didn’t have the bandwidth to handle any more search requests.

As a multi-store platform, Netshoes IT knew that if the new platform experienced any issues, it would affect all partner stores in the same way. For example, if the platform was unable to handle all search requests, it could lag or even crash all 32 Netshoes site search capabilities. “Netshoes operates entirely online, that’s why we need to ensure the best search results to our customers,” explains Marcelo Vidu. Reduced search capabilities would be disastrous for sales during a key retail shopping day. Netshoes knew they had to address supporting the store transaction volumes by totally adapting both infrastructure and data structure to support the bulk queries that would be carried out simultaneously on the platform.

Recognizing that their current Netshoes BR platform would not handle the web traffic they expected on Black Friday, they contracted Elastic Consulting to not only help get them into production, but also to provide best practices for optimization, tools for monitoring, and to upgrade Elastic clusters.

Elastic Consultants came onsite to work with Netshoes engineers at their headquarters in São Paulo, Brazil. They worked together to adapt existing architecture, data model, and machine sizes to handle more traffic.

The Netshoes team directly reported all the developments and learnings that came from working with Elastic Consultants during migration—including performance metrics, improvement in search results, and scalability of handling Netshoes traffic — to Andre Petenussi, CTO at Netshoes.

As a starting point, Elasticsearch Rally was implemented to standardize performance testing and to observe monitoring metrics and documents that were indexed for search. At the end of the testing and observations period, Elastic Consultants helped deploy a new POC cluster and indexed production data into it. Then they optimized sample production queries, and ran them against the POC cluster as tests. Lastly, based on the test results, they looked to find the best configuration for a new production cluster.

Elastic Consultants suggested a few critical improvement points to reach the required performance for the new production cluster:

1. Upgrade from Elasticsearch 2.4 to 6.1 to improve query performance
2. Monitor Elasticsearch from the cloud
3. Eliminate fuzziness between Spanish and Portuguese queries

In order to monitor from the cloud, Netshoes had to upgrade their Elasticsearch version from 2.4 to 6.1 to be able to use the latest monitoring features included in their platinum subscription. Kibana’s monitoring plugin gave Netshoes visibility over both cluster latency and overall health, information that was important for optimizing performance. Monitoring from the Kibana dashboard also provided metrics such as search rate and the number of indices and documents. Additionally, the upgrades from 2.4 to 6.1, which includes the latest versions of Elastic features, greatly improved performance, security, and stability.

Netshoes and Elastic Consultants also realized that using Master and Ingest nodes together resulted in better performance, and that keeping data nodes on dedicated machines improved it as well. Data ingestion was also changed. So every time they needed to index a large amount of data, a new index was created with a unique name. This new approach has kept the cluster more reliable, stable, and has decreased query response time for Netshoes’ customers.

Adjusting fuzzy queries was also a high priority to Netshoes to provide a great customer experience on their site. Netshoes largely operates in three different jurisdictions: Brazil, Mexico, and Argentina. However, indices for stores in all three were configured to use their custom ns_brazilian Brazilian Portuguese analyser, despite Mexico and Argentina largely being Spanish speaking countries. This had gone unnoticed until recently, due to the close relation between Portuguese and Spanish. Elastic Consultants advised how to resolve this issue through the use of layered templates.

Elastic Consultants tested the previous production cluster and compared searches with and without fuzziness. With “fuzziness”, returned a high amount of false positives, defined as: the additional documents returned above the number of results when searching with the correct spelling. In comparison to “without fuzziness”, returned the accurate number of documents with the correct spelling. The tests were run against commonly misspelled Brazilian search terms “camiza” versus “camisa” and “camizeta” versus “camiseta.” Elastic Consultants recommended addressing fuzziness at a factor of 1—meaning that if the incorrect term was searched for, Elasticsearch would produce relevant alternative results with one letter different. This is extremely important for a customer’s overall e-commerce shopping experience. That’s because relevant results will always be rendered for the online shopper who incorrectly spelled a query. The alternative being many irrelevant results or no results at all for an incorrectly spelled query.

In addition, index templates in Elasticsearch have an “order” parameter which contains an integer value to compare against conflicting templates, with the highest value winning the conflict. Netshoes created layered templates and configured them to use a two-letter country code of each index name ar, br, mx. They also created appropriate higher order templates to apply language-specific settings while having a lower order template handle index settings common to all. This action of taking into account geographical location has largely helped increase search relevance.

Today, the new platform has increased search-based support by six times, provided data control, and it fully supports the bulk of simultaneous queries performed on Netshoes’ platform throughout Latin America, including their partner stores. After migration to an Elasticsearch based platform, Netshoes now has three developers dedicated to search, improving the support process, and increasing search reliability and availability. In preparation for the 2018 Black Friday, Netshoes’ new Elastic Stack powered platform was tested to be able to handle six times the web traffic compared to in 2017. It enabled a a better search experience, and provided complete search information to customers in 150 ms.

Netshoes initially ran the Elastic Stack on-premise, but have since migrated their e-commerce business to the cloud via Microsoft Azure. Thanks to the monitoring provided from their platinum subscription, the team is now able to focus on more important tasks pertaining to the core business, in addition to developing new features rather than handling infrastructure, support, and operations themselves. With Elasticsearch running on Azure, Netshoes has been able to grow and scale their infrastructure faster than they could with an on-premise datacenter.

“Netshoes’ next steps include streamlining processes, increasing automation, and reducing delivery time of new business functionalities, as well as developing a smarter platform aimed at end customers, all powered by the Elastic Stack.” says Vidu. They are planning on upgrading to the Elasticsearch Service in 2019 to develop a centralized logging platform.