Today’s users, customers, clients, and buyers are notoriously impatient. They demand new experiences that engage and inspire. They also want them delivered at light speed. Because of these demands and expectations, applications and data stores have evolved in order to provide these types of “I want it now” experiences.
We as customers, consumers, and end users take personalization for granted. For example, we know that many travel operators will present car rental offers to go with the flight we just booked. Spotify will always know which song to play next and which types of playlists or artists we might like to listen to next.
While users have the luxury of taking these real-time personalized experiences for granted, application developers do not. The ability to instantaneously perform the queries that provide the results that fuel real-time personalization is critical not only in avoiding angry, impatient users but, often, in generating more business. After all, there’s no point in presenting car rental suggestions (based on a resource-consuming analysis of a user’s travel itinerary, attributes, and past and present buying behavior) if the user has already left the app.
Customer-facing applications, in particular, have a high bar to meet. With internet latencies at approximately 50 milliseconds round trip, application processing, data access, and response must all occur very quickly. To achieve this at scale, databases must be capable of delivering sub-millisecond response times under conditions of any load while simultaneously offering highly personalized user experiences.
NoSQL Databases to the Rescue for Personalization
Tracking and storing all of this data is possible thanks to NoSQL data stores that can handle disparate types of data from more sources.
Luckily, that is where improvements to offerings in databases and data stores come in. Online platforms and e-commerce sites in particular have experienced the explosion in data and data types that NoSQL databases handle better than structured relational databases. There is no true need for user data or catalog data to be forced into a rigid, structured schema. NoSQL databases allow app developers to be very flexible to the point where they almost just grab it and stuff it in there for later processing and uses in a variety of ways. (As a former relational DBA for many years, that is still hard to type.)
Bringing customer data into a single database shared by multiple applications has helped simplify the immediate architecture and eliminate some silos that used to happen when each application had to have its own specific type of database.
Having varied application types (such as e-commerce, CRM, billing, etc.) able to use the same data store can make life much simpler for developers and for the DevOps and support teams responsible for them.
Most modern apps take in unstructured data such as text fields, chat strings, dates, events, actions, posts, video, email, etc. This data is delivered in a variety of formats including mpegs, jpegs, JSON, etc. With these varied data types, there is no reason to suffer the additional higher latencies that usually accompanies the cross-table joins and heavy queries of relational databases. Because of their lighter weight data models and the ability to scale across multiple servers to provide additional resources, NoSQL databases can achieve much higher performance than traditional relational databases when it comes to the real-time personalization needs of businesses. This is especially true with e-commerce and online shopping sites with their often huge and diverse online catalogs and product offerings.
Flexible Data Structure
NoSQL databases, like MongoDB, offer more flexibility than relational databases, but not all NoSQL data stores are created equal. True performance boosts occur when an app can take advantage of being able to support a variety of incoming data types including those that are unstructured, semi-structured and even yes, structured. Even applications with varied read-to-write ratios can take advantage of this simplified structure requirement to write quickly to the database, add fields on the fly (even if coming from multiple sources), perform aggregations and come up with more complex result sets along the way. For example, many product catalogs use MongoDB as the main data store because it functions well as the incrementer/decrementer and can handle large usage spikes during big events.
Personalization for Retail
Retail businesses rely heavily on planned events, such as Black Friday or the day after Christmas, to generate and maintain their business and market share. If the product catalogs are not quickly updated, they are losing sales. Unhappy customers will look for other places where inventory is more reliable and available more quickly.
Keeping track of a customer’s cart contents and even hanging on to a pointer for what they deleted, helps personalize a user’s profile and allow them to find similar brands and offerings that they might not be aware of and would not even have known about.
Redis for Caching
Using an in-memory data store, such as Redis, can deliver faster data access than disk-based architecture can. Caching plays a critical role when your application is tasked with capturing real-time events at scale and then subsequently rendering personalized experiences based on these events.
Using the Strengths of Multiple Data Stores
With Redis, you can take the cached results from your application queries and keep the most used query. Think “top searches” such as “Top 3 Most Purchased Basketball Shoes”, “Top Watched Movies on Netflix”, “Top 10 Best Burgers in the United States”… you get the picture. Once you collect the aggregated results in one data store, you can place them in an Elasticsearch index. You can then cache these results in Redis and serve the result sets up so fast that customers buy even more. All of this can be targeted specifically to individual customers so that they do not have to waste their time plowing through pictures of products that they could care less about.
Sorted sets, sets, hashes, lists, strings, bitmap, and hyperlog are all examples of data structures designed to not only more elegantly store variably structured data, but also perform complex analytics on the data via built-in operations. The more data structures and built-in operations you have at your disposal for current and future personalization needs, the more network and computing overhead you can eliminate while also radically simplifying application development.
To take the personalization experience and develop the overall platform even more, look to improve your customers search experiences by decreasing the amount of time it takes to bring back that first result when your users start typing in your website or application search. Using the super fast capabilities of Elasticsearch when handling fuzzy text searches and incomplete text searches can help you get ahead of competitors who are still relying on and slowed down by the joins in the relational world. Additionally, with Elasticsearch’s rank and scoring nature, you get improved relevancy in your queries and the ability to scale quickly by adding additional nodes.
Modern applications must generate engaging and inspiring personalized experiences. And they must do it without sacrificing a drop of performance, and keep doing that as they data stores grow in size.
Fortunately, with the right mix and usage of data stores, these objectives are not mutually exclusive. Polyglot Persistence! Use the right tool for the right job. If you’d like to speak with our DBAs and customer data experts, contact us so we can talk about the right database for your workload.