Uncategorized

What is Polyglot Persistence?

By April 30, 2018 August 18th, 2022 No Comments
Polyglot Persistence Blog Header

Polyglot persistence refers to using different data storage technologies to handle varying data storage needs. It’s an offshoot of polyglot programming, which means using different programming languages to build an application. Simply put, it’s an application that uses more than one core database technology.

polyglot persistence

Your parents may have stressed that it’s important to use the right tool for the job. It’s just as important to use the right database for the job, too. Each database has strengths and weaknesses. Each database has its own idiosyncrasies. Now that there are literally hundreds of databases out there, it’s hard to keep up.

Managing Your Data Storage Needs

Monoglot was (and still is) fine for simple application that uses one type of workload. However, applications quickly become more complex.

A simple E-commerce platform uses:

  • Session data (Add to basket)
  • Search engine (Search for products)
  • Recommendation engine (Customers who purchased this item also like this)
  • Payment platform (Geolocation service)

Implementing polyglot persistence allows you to fuel applications with data from different storage types. You could pull completed financial transactions from Oracle for your reports while tracking user information in MongoDB and letting Redis handle cache.

Implementing Polyglot Persistence

Many complaints referencing the complexity of building out polyglot persistence come from inexperienced engineers forcing storage solutions into projects they’re not right for. That leads to the creation of numerous support processes and an ever-growing monster architecture.   

Consider All Polyglot Persistence Architecture Options

Planning things out properly from the beginning eliminates 90% of potential issues. What are you trying to accomplish? Is it an integral business function? Does it make sense to start transitioning something a large portion of the company depends on to this model?

Using the Right Data Storage Technology

Has your team created a polyglot persistence architecture before? It’s challenging to properly leverage the right data solutions without experience designing this type of model. How do you know if you’re even starting with the right project?

Data Storage Types

Organizations often end up using a number of different data sources to feed their applications. Let’s go over some common storage options:

Transactional (RDBMS) Databases – They provide data integrity by organizing information into tables with rows and columns. You retrieve data using syntax statements called queries. Many large organizations make heavy use of popular options like SQL Server and Oracle. One drawback to transactional databases is the difficulty in scaling as storage needs grow. Searches slow as data gets more dense and complicated to access.

Non-Transactional Databases – People often use the term NoSQL to describe these databases. They mark a shift from the use of tabular RDBMS systems, allowing for increased scalability and faster searches. They also pair well with search and analytic engines like Elasticsearch.

Non-Transactional Storage Types:

Key-Value – These databases use a key that points to the storage location of a piece of data. Some use hash-tables, while others like Redis hold its pairings in memory for quick access.

Document –  These consist of key-value pairs compressed into a document that allows for structure by using some type of encoding like JSON or XML. Popular open-source versions include MongoDB and CouchDB.  

Column – Data gets placed into grouped columns instead of rows like in an RDBMS structure. Cassandra and HBase use this model.

Graph – Uses graph structures along with nodes, edges, and properties to organize and retrieve data. Neo4j is an example that’s growing in popularity.

Getting polyglot persistence right means knowing which data options work best for your application needs.

Questions to Ask

Data structure questions:

  • Does your data have a natural structure? Is it unstructured?
  • How is it connected to other data?
  • How is it distributed?
  • How much data are you dealing with?

Access pattern questions:

  • What is your read/write ratio?
  • Is it uniform or random?
  • Is reading or writing more important to your application?

Organization need questions:

  • Do we need authentication? What type?
  • Do we need encryption?
  • Do we need backups?
  • Do we need a disaster site?
  • What level of monitoring will be required?
  • Which drivers are needed?
  • Which languages will be used?
  • Do we need plugins?
  • Will we use third party tools?

Bringing in the Right Help

Many companies would benefit from bringing in experts in polyglot architecture to help with these decisions. That way you avoid ending up with additional maintenance loads due to overly complex architecture. Some popular open-source database vendors offer management services. But will they let you know if their own data source isn’t right for your needs?

It’s best to reach out to a company that understands the strengths and weaknesses of different data options to get you started with polyglot persistence. ObjectRocket gives you a 360-degree view of numerous database options applicable to your business needs. We bring in DBAs and engineers best suited to take your big data solutions to the next level.