Strolling along a main avenue in a European city, I recently came across a big scene of contemporary retail – the spectacle of a crowd of people outside a shop window, showing such a degree of excitement, cameras held aloft, as is usually only seen before the Mona Lisa in the Louvre.
The reason was another form of modern 'art’: a half-naked model sitting on a chair, surrounded by piles of clothes (the shop was a fashion franchise).
Whether most of the people were gathered at the window because of the nakedness of the model, or the novelty value in this street of identical shop windows, or due to the brand’s message (to make us aware of the excessive amount of clothes we own), it was obvious that in any case one thing had been effective: an eye-catching image.
From the outset, online sales have had to cling to visual appearance as the most persuasive argument available to convince buyers, being unable (as yet) to make use of our other senses.
However, this basic notion is not entirely true: selling and buying has always been a process based on that spark of falling in love with something you see. No-one feels persuaded to buy a briefcase just because of a friend’s description of it, or decides to buy a dress because of the softness of its fabric (what about the cut, the length and the size?), or to eat a mouthful of food just by smelling its aroma (but what if it’s an insect?).
→ Looking for the perfect shop window? Tips for fashion marketers
To a large extent, all of those ideas are somewhat flawed. There are many present-day experiences that challenge our dependence on the sense of sight and reveal how unfairly constructed the world is for blind or visually impaired people. However, both for people with a well-developed sense of sight and for retailers who are not yet able to benefit from immersive technologies, seeing something continues to be the key factor.
“50-80% of e-commerce retailers new customers come from visual search. As humans, we buy by experience, and visual is one of the closest experiences we can have to trigger the imagination. These are already mature and proven buying channels for customers. It's absolutely important to leverage it to grow your business.” | David Jaeger, Result Kitchen Founder
The basic reason is very simple: for the human brain, the step that comes before desiring something is looking at it, and it only takes 13 milliseconds to recognise an image. Since most purchases that follow from online searching, are based on a desire or a whim derived from a more useful purpose (I was looking for a pestle and mortar, but now I've convinced myself I want a model that’s in green marble), both still and moving images are the most valued raw material in online sales.
However, the sector needs to go a step beyond such traditional methods as leafing through a catalogue, watching an ad on TV, or taking note of something advertised on an underground poster. Until recently, online searching and buying was not very different from those methods, although it did entail the extra comfort of being able to pay and order instantly. Even if results were visual, searches remain textual and passive: the user has no control over precisely what images he wants to be shown.
Visual search has come along to solve that need and to give online shoppers more control, personalisation and interactivity. Finding the perfect product will no longer be like playing “Where’s Wally”.
A few years ago, the novel option of combining aspects of fictional media with online shopping was hailed as revolutionary: what if, while watching the latest James Bond movie, you could buy the Rolex worn by Daniel Craig just by clicking on the screen?
Although this idea has given rise to a large number of applications which recreate the clothing outfits of characters from a series and allow the buyer to locate such garments, in reality their freedom of choice remains minimal.
On the one hand, this strategy means that a content creator has to depend on such brands as might have offered to supply their products for the film or series (classic positioning); while on the other hand, although the product shown might have convinced the user to buy, he or she will then be shown no alternative to that specific model of that particular brand.
The real revolution in visual search is that the user gains power over the results, in a more refined way than through written or dictated words. The value of enriched data for searches and online retail both now and in the future is indisputable: well-structured content facilitates positioning in search engines and improves the experience to make it more personalised and precise. And this product content will eventually be reflected in key images of visual searches.
→ Related: 10 ways to improve your customers shopping experience
In simple terms, visual search is the ‘Shazam’ of photographs: it empowers the user to carry out an online search based on a reference image. And the positive thing about this method is that it no longer restricts the customer to buying only the Rolex MI6 collection. Users can find products similar to that in the image by applying different filters and ranges, and retailers can see an increase in new visitors and buyers thanks to suggestions for products related to other brands.
All you need is an image whether snapped with your mobile phone camera, downloaded from the Internet, or saved on social networks. By entering it into a program or search engine, the user finds out what products are available that match that image exactly, or that are similar, cheaper or more expensive.
This system takes advantage of the buying impulse, because the sooner you find a product that’s available, the more likely it is that you’ll decide to buy it without pausing to think about similar, perhaps even more convincing, options. By using visual search, the shopping journey is shortened and becomes more efficient, eliminating the restrictions and frictions of the classical method.
It puts an end to the noting down ideas for later, browsing, checking and comparing manually until the user feels frustrated if there are no matching results; or if they don’t know which shops or marketplaces to make comparisons across, or don’t know how to express properly in writing a description of the product they want. According to some studies, visual searches result in twice as many purchases as text searches, while 62% of new-generation users prefer visual search; it's time we adapt to them.
The year that’s usually marked as the official starting point for effective visual search is 2017. That year, software companies specialising in visual search tools and extensions appeared on the market, as well as plugins on well-known platforms, starting with that kingdom of imagery: social media.
The limitation of visual shopping using badges is, as we mentioned earlier, that they do not facilitate comparison. However, many apps and online tools that develop artificial intelligence technology in the service of visual searches can follow several options at once, as do the flight comparison tools.
Most of them work best for interior decoration, clothing garments and food: that popular triad on social media. But as the technology is refined, so will applications spread. What if someone can find on Tinder, a person resembling the model he saw and photographed in a shop window? Ethical questions also need addressing.
Retailers love to have the backing of figures when a new technology emerges, and in the case of visual search they are unparalleled: Gartner predicts a 30% increase in sales for those who adopt this technology before 2021, and there are already success stories with an increase of between 20% and 50% of sales, such as Forever21 and Nike, according to Oliver Tan, CEO of Visenze.
Undoubtedly, for the fashion industry visual search is an essential accessory, along with shoppable images, new 3D and augmented-reality technologies that create virtual fitting rooms to try for size, and digital assistants such as chatbots.
→ Fashion marketer? Follow these 7 digital trends
Finding the key product from a photograph is a big step forward for the user, but for retailers even more attractive is the possibility of related content, whether in search engines or in the internal visual search options of the website or app.
“An advantage of visual search is that it relies entirely on item appearance. There is no need for other data such as bar codes, QR codes, product name, or other product metadata.” | Brent Rabowsky, Amazon Web Services Machine Learning Specialist
Of course, this technology has its limitations too. The results are not yet 100% reliable, because some refinements are missing, such as being able to apply filters that avoid showing suggestions for cheap copies; neither can the whole Internet, nor excessively broad catalogues, be searched, apart from there being certain ethical implications.
It would be questionable if someone were able to upload the photograph of an unknown person, snapped on a street or in a café, that resulted in it being stored in some online database or record. This may be acceptable where models working in the public sphere are concerned, such as the app that Tommy Hilfiger offered in 2017 to identify garments from catwalk shows, but its use with anonymous people or even children could lead to a lot of problems.
Visual searches may sound great, but how can you ensure that products from a catalogue or store will be included in the results? For the moment, the recommendations are not very different from those that should already be applied in order to improve the positioning of images in text searches.
“Now, retailers can use computer vision technology to interpret the style of their inventory, as well as its size and color. That creates entirely new ways to curate an e-commerce store and, in turn, a significantly better experience for the customer. For their part, the customer can find products in those moments when they would previously have thought, 'I know what I want, but I don't know how to describe it.’ | Clark Boyd, Digital Marketing Consultant
As we have already indicated, many platforms and brands develop their own internal visual search system in their web pages and apps, although the product content must always be prepared so as to appear in the results of other search engines.
Some systems are very simple, such that by photographing a product the app leads to the product profile of the website, such as Lush. But by using the metadata of an image, any system based on artificial intelligence can detect which visual results are really relevant, based on similarities of form, colour, pattern and other elements that are, so to speak, eye-catching.
For example, the well-known Macy's, Mark & Spencer or ASOS chains allow you to search for products from their catalogue based on any photo that you upload to the app. However, developing this type of proprietary or in-house technology can be quite expensive, especially if the catalogue is very extensive. For this reason, many brands use Pinterest Lens as a substitute for their own visual search engine, such as the Target clothing brand.
→ Don't feel a bit lost: Start your own product management strategy here
Developments like these by specialised companies are the most interesting, as they go a step beyond searching for copies of an image, such as IBM Watson or Clarifai. For example, Synthetic's Style Intelligence Agent (SIA), developed with Adobe Sensei, is a discovery service that uses artificial intelligence to help the user complete their outfits of clothing and discover many possibilities that might complement a single image.
This means that the system can not only look for results similar to a photo, but even deduce (or recognise) patterns, and search for products across different categories; such as clothing inspired by the style of certain artistic paintings, interior design relating to images of natural landscapes, or products that match other items they already own. Do you want a dress that resembles Van Gogh's Starry Night, an armchair the colour of the Serengeti, or a washing machine that matches your red toaster? Just upload a photo, choose the desired filters and the robot will do the analysis for you.
While these services may be incorporated into brand catalogues, visual search apps such as CamFind are already appearing that will search multiple brands and results online: a really interesting option for buyers who are tired of comparing manually, on Google and Amazon.
Other more futuristic options, that we shall be seeing from 2020 onwards according to technological experts, are scanners to compare prices between different stores (as Walmart is currently testing) and mirrors in physical shops capable of measuring the dimensions of a customer to offer them the best suggestions of clothes in stock.
In both the online and offline worlds, the trend seems clear: to save the customer the tedium of searching digital and physical shelves, and to personalise the service of recommendations as much as possible.
For retailers, this push forward of artificial intelligence into visual search tools signals the need to be quick in its implementation, because unless the AI system can scan 99% of an inventory, the results will not be as accurate nor will the option be as attractive to the buyer. To link so many articles to a new system, companies must act in time and with sufficient organisation.
“One of the most important trends in marketing is personalization. The ability to arm your customers in a retail setting with visual search is an opportunity to take personalization to the next level. So embrace it. Invest in it. And unlock the power of both personalization and visual search before the competition does.” | Ross Simmonds, Digital Marketing Strategist
The applications of these technologies of visual search and image recognition are endless, from recognising plants to identifying people in security systems, but without doubt it is in the retail sector that it has become a favourite talking point.
The advantages of implementing visual search in a website or app, apart from appearing in visual search results, are outstanding: greater customer loyalty, increased sales, and improved benefits in the short and medium term. The key is to adapt the type of visual search to the customers’ most common goals and to the manner in which they tend to search, use and buy products. And in this new positioning, an image can be worth more than a thousand keywords.