0

Say we're making a simple photo app. This link says that we can store the image in S3 and now have a URL. Great. Next, we need a mapping of a UserID to the many images they've created. For this, it recommends Cassandra where the key would be UserID and the value would be the list of PhotoIDs stored in different columns.

However, why can't we use a document DB like mongo instead? It can have something like:

{
    “UserID” : abc
    “PhotoIDs” : {
           url1,
           url2,
           url3,
            ..
      }
}
    

Or a persistent K-V store like DynamoDB?

What is Cassandra's column-based storage giving us here that these don't?

I realize that the unbounded nature of photoURLs could be an issue. Say a user has 10,000 photos, adding one to this would mean fetching the 10,000 item large document, adding to it and then updating the document. But does cassandra solve that? Would it be able to have 10k columns for that one prolific user?

2 Answers 2

5

Cassandra right now, is not a wide column store - it was before CQL that brought the fixed schema, etc. You can still model necessary functionality using just a number of the columns in the table. The main thing that you need to remember about Cassandra, is that all data modeling for it is starting with queries in mind - how would you fetch data from Cassandra. For example, if you want to display photos organized by user & time, then you can come with following schema:

create table photos (
  userId uuid,
  photoId timeuuid,
  photoUrl text,
  primary key(userId, photoId)
);

In this case, you can do the following:

  • easily add a new photo without reading all data upfront - just add an unique photoId
  • read all photos for user by doing select on userId only
  • read a specific photo - by doing select on the userId + photoId
  • read photos in given time range - it's possible because photoId is time-based UUID, so you can do queries like this: select * from photos where userId = ... and photoId > minTimeuuid('startTime string') and photoId < maxTimeuuid('endTime string');

Although, in given case, we'll need to think about other things as well - for example, how many photos we may have per user? If we may have millions of photos, then we may need to introduce some additional bucketing, so we won't get huge partitions with data. In given case, we may use the year as an additional partition key column, changing the primary key to primary key((userId, year), photoId), and adjusting our queries accordingly.

So when we compare with other systems, we have following advantages:

  • for K/V store you need to fetch each key/value pair separately, doing a lot of requests. In Cassandra you can fetch all data, or slice of data with one query
  • for document store, you may need to fetch all photo IDs even if you need one, and maybe also need to fetch them just to insert only one item.

I recommend to take DS220 course (data modeling for Cassandra) on the DataStax Academy.

Also, besides only data modeling thing, you need to think about non-functional requirements. For example, Cassandra gives you out of box the following:

  • No single point of failure - there is no master/leader node, etc.
  • built-in replication of data, allowing to build clusters spaning multiple geographical regions
  • linear scalability (with correct data model, of course)
4
  • Thank you! and this makes a lot of sense, A question about the photoID, can't we just have it be UUID and avoid complexity, and use create_date as the clustering key to sort and retrieve in order? Jul 6, 2020 at 16:56
  • theoretically - yes, but for retrieval of the single photo you'll need to know both timestamp & ID
    – Alex Ott
    Jul 6, 2020 at 17:04
  • I'm a cassandra noob, so you might have to hand hold me a bit here. Why do I need timestamp for a single photo retrieval? Isn't the photoID enough? Jul 6, 2020 at 17:53
  • this is how Cassandra works. To fetch single item, you need full primary key. You can move ID into 2nd position of PK, and add timestamp after that - but then you won't be able to make a query for range of time. By using timeuuid we're combining ID & timestamp together
    – Alex Ott
    Jul 6, 2020 at 18:00
5

Cassandra is NOT column-oriented storage

From Apache Cassandra project on GitHub,

Cassandra is a partitioned row store. Rows are organized into tables with a required primary key.

Partitioning means that Cassandra can distribute your data across multiple machines in an application-transparent matter. Cassandra will automatically repartition as machines are added and removed from the cluster.

Row store means that like relational databases, Cassandra organizes data by rows and columns.

So if you have a table Photos:

userId   creationOfTime     photoURL
  1          12345          abc.def
  2          12356          url.url

Cassandra stores the above data as

"Photos": {
           row1 : { "userId":1, "creationOfTime": 12345, "photoURL":"abc.def"},
           row2 : { "userId":2, "creationOfTime": 12356, "photoURL":"url.url"}
           ...
          }

Database Options

Note the data schema may have not to do with the database option, so a schema like the above example serving as an indexing to the objects in S3, can apply to Postgres (even if it's SQL), DynamoDB, Cassandra, etc. Postgres supports JSON data type so the schema can be either

CREATE TABLE IF NOT EXISTS Photos (
  userId TEXT,
  timeOfCreation LONG,
  photoURL TEXT,
  primary key(userId, timeOfCreation)
);

or

CREATE TABLE IF NOT EXISTS Photos (
  userId TEXT,
  photoProperty JSON,
  primary key(userId)
);

where photoProperty would be a map of timeOfCreation:photoURL. While the data update may not be an issue, we'd prefer the first schema because it supports time range query --- in Postgres or Cassandra we use something like select * from ... where userId = ... AND timeOfCreation >= ..., in Dynamo it is equivalent but it supports sort key explicitly (ref) as part of primary key so again we can query for photos within a given time range.

The option of database relies more on the scalability and performance requirements.

Wide column database

Referring to the definition here:

Its architecture uses (a) persistent, sparse matrix, multi-dimensional mapping (row-value, column-value, and timestamp) in a tabular format meant for massive scalability (over and above the petabyte scale).

From the definition, wide-column storage does not require a defined table structure --- rows in a wide-column database don’t need to have the same columns, so it enables developers to dynamically add and remove new columns without impacting the underlying table.

Haven't thought of a real use case in your photo storage example, but for wide column, basically the schema is similar with entity-attribute-value model and looks like:

userId   creationOfTime   some_attribute  other_attribute      photoURL
  1          12345            some_value                       abc.def
  2          12356                          some_other_value   url.url

looks like your use case does not benefit from wide column very much, and thus wide column has not to do with if Cassandra/Dynamo JSON should be considered.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.