Discovery, Search, and the Hunt for Buried Treasure

Image via

Co-editor of the 18th-Century French Encyclopedia, rebel against conventional narrative style, renowned critique of art and drama, predecessor of complexity theory, and advocate of feminine sexuality, Denis Diderot is one of my personal heroes. This Enlightenment polymath devoted much of his philosophical energy to championing Empiricism – the school of thought that holds that knowledge is rooted in observations we accumulate in life – over the then prevailing school of Rationalism – which holds that knowledge is rooted in hard-wired, logical patterns of thought we are born with. Rationalists build clean, coherent theories and search for data to support their conclusions; empiricists trade elegance for complexity, allowing observations to lead them astray into discovery, unearthing knowledge they didn’t know they were looking for when starting their quest.

To illustrate his preference for inquisitive empiricism, Diderot repurposes one of Aesop’s fables in his Thoughts on the Interpretation of Nature, published in 1754. The fable tells of a father, on his death bed, who advises his children that there is treasure buried in his field, but that he does not remember where. The children dig and scourge the field to find the treasure, but to no avail. The following year, they continue their search but with sharper tools, convinced the treasure must be buried deeper in the ground. Eventually, one of them sees some shining fragments and realizes he may have discovered a mine. So the children shift tactics. They give up looking for treasure to focus their efforts on exploiting the mine, which yields plenty. For Diderot, the children start as rationalists obsessed with solving a particular problem which was likely unsolvable and ended as empiricists, who “come to make discoveries more important than the solution itself.”

These debates between rationalism and empiricism still take place today, although in slightly different forms to match different stakes and discussions. And, with a bit of imaginative license, we can draw analogies between these two approaches to knowledge and the use of contemporary technologies.

Consider, for example, the distinction between search and discovery as tools for finding and curating content on the internet. Search tools provide exact character matches between the input entity one searches for and the collection of content one searches on; input “horse,” and the engine will scan content to find the word “horse” and return pages that contain that word as the result set. Multiple organizations are hard at work to refine these tools to intuit the intention of the searcher when conducting the search. The assumption is that, through our habitual use of search engines like Google or Yahoo, we’ve started to develop our own niche language for how we think queries are structured. For example, when an expecting mother is looking for a stroller for her son, she does not transcribe her literal question (“where can I find the best stroller for my son?”) but rather inputs a truncated phrase to mimic how she expects the search engine functions to get the results she wants (“best strollers”). If the engine is right, search can be powerful, extensive and precise. But the extent of the result set is always limited to what the searcher thinks he or she is looking at the outset. Search is the tool to help the children in the fable dig through the field to find the buried treasure.

Discovery, by contrast, extends the parameters of our initial inquiry to unearth content we may not know we’re looking for when we start. Words are not absolute, independent units with their own intrinsic meanings, but relative, dependent units whose meanings blossom in context. For each word, therefore, there are multiple related words that create clusters of meaning that index a topic, theme or context. To return to the example of “horse,” we might situate horse within the cluster of words related to the practice riding (tack, bit, saddle), the cluster of words related to similar species (donkey, zebra), the cluster of words related to sociology and demographics (aristocrat, noblemen, polo). Someone initially searching for “horse” may be interested in any one of these clusters. While a curating engine needs further information to discern the intended cluster, it can use subsequent activity to provide more accurate and relevant results going forward. Once given the cue to find content in a particular cluster, a discovery engine can return vastly different results than search: our horse-lover may start an inquiry with the word “horse” and end up finding a fascinating body of knowledge about Lully’s 17th-century ballets. To return to our fable, discovery is the tool that helps the children find the plentiful mine, riches greater than the treasure they originally sought.

The applications for content marketing are powerful. In today’s world, branding strategies have shifted from inundating repetition of single images and refrains to distributing varied content through vehicles that best align with individuals’ particular habits, interests and preferences. Like words, people’s interests are not discrete and absolute; they are tied into an integrated whole, each topic a Lilly pad connected to others by a root system of analogies, associations and experiences. Simply put, Trapit’s discovery engine enables content marketers to enrich their corpuses, filling them not only with precisely defined content indexed by search, but also with the vast body of related topics that extend horizons, providing consumers the novelty they want to retain loyalty to a trusted brand.

So often in life, it’s when we’re not intently focused on searching for something that we open ourselves up to discover the beautiful surprises that end up providing us the most meaning and happiness. A woman went shopping for shoes, and came home with her favorite dress (forget the shoes); Proust took a bit of a cookie and found one of the greatest novels of the 20th century; a man started a conversation with a woman on a plane and ended up marrying her, the love of his life. Knowledge we don’t yet know we’re looking for is out there to be discovered in the endless, dormant mines of the web.

– Kathryn Hume

Kathryn Hume leads marketing for the Risk Practice Group at Intapp, Inc, a software company that provides business operations technology to law firms. She holds a PhD in Comparative Literature from Stanford University and focuses on the intersection between law, the humanities and new technologies.