Data model in Cassandra and proper deletion Strategy

问题

I have following table in cassandra:

CREATE TABLE article ( 
id text, 
price int, 
validFrom timestamp,     
PRIMARY KEY (id, validFrom)
) WITH CLUSTERING ORDER BY (validFrom DESC);

With articles and historical price information (validFrom is a timestamp of new price). Article price changes often. I want to query for

Historic price for a particular article.
Last price for an article.

From my understanding, I can solve both problems with following query: select id, price from article where id = X validFrom < Y limit 1; This query uses article id as restriction, query uses the partition key. Since the clustering order is based on the validFrom timestamp in reversed order, cassandra can efficient perform this query. Am I getting this right?

What is the best approach to delete old data (house-keeping). Let's assume, I want delete all articles with validFrom > 20150101 and validFrom < 20151231. Since I don't have a primary key, this would be inefficient, even if I use an index on validFrom, right? How can I achieve this?

回答1:

You can use external tools for that:

Spark with Spark Cassandra Connector (even in the local mode). Code could look as following (note that I'm using validfrom as name, not validFrom, as it's not escaped in your schema):

import com.datastax.spark.connector._
val data = sc.cassandraTable("test", "article")
   .where("validfrom >= '2020-07-28T11:50:00Z' AND validfrom < '2020-07-28T12:50:00Z'")
   .select("id", "validfrom")
data.deleteFromCassandra("test", "article", keyColumns=SomeColumns("id", "validfrom"))

use DSBulk to do find the matching entries and output them into the file (output.csv in my case), and then perform their deletion:

bin/dsbulk unload -url output.csv \
  -query "SELECT id, validfrom FROM test.article WHERE token(id) > :start AND token(id) <= :end AND validFrom >= '2020-07-28T11:50:00Z' AND validFrom < '2020-07-28T12:50:00Z' ALLOW FILTERING"
bin/dsbulk load -query "DELETE from test.article WHERE id = :id and validfrom = :validfrom" \
  -url output.csv

回答2:

To add to Alex Ott's answer, this comment of yours is incorrect:

This query uses article id as restriction, query uses the partition key. Since the clustering order is based on price, cassandra can efficient perform this query.

The rows are not ordered by price. They are ordered by validFrom in reverse-chronological order. Cheers!

来源：https://stackoverflow.com/questions/63121594/data-model-in-cassandra-and-proper-deletion-strategy

标签

cassandra