问题
I want to store complex objects such as a DataTable or Dataset, etc, in Redis. I have tried to serialize them as a BLOB object using JsonSerialize, but it takes too much time. Is there any other way?
回答1:
Unfortunately when working with large data sets it will always take time to serialize and deserialize the structure. DataTables in particular are fairly complex objects, as they have rows and columns which often have a lot of meta data attached to them - even when it appears to be a basic table.
DataTable vs List<POCO>:
Consider whether you really need to be serializing as a DataTable. Could you create a simpler POCO and serialize a List<YourRecord>? In other words, if you don't need extra attributes on fields and columns and you can serialize to a simpler format, it's likely quicker, and more space efficient in the cache; and then restore to a DataTable if necessary.
Another option is to split the DataTable into smaller sets, that you serialize and store in smaller parts. You may find this more performant. You should be able to benchmark this.
Benchmark:
Ultimately your Redis cache should be an improvement over the time taken to re-query the data source. You use the term takes too much time, but if it takes 2 seconds to get from the cache vs 8 seconds to query the data source then that is a significant boost. But the only way to be sure is to benchmark.
Setup your environment so you are only running necessary tools. Don't perform other tasks while running the benchmarks, so you don't introduce any bias.
Record the time it takes to serialize a
DataTable. Perform this action many times and average.var start = DateTime.Now; // Serialize var duration = DateTime.Now - start;Experiment with different sizes of
DataTables and see if you find an acceptable time.Try a different serialization library, such as JSON.NET. While it's nice to keep it all ServiceStack, this can help you determine if it's a shortcoming of ServiceStack.Text or just an issue with the large dataset.
Repeat the process for deserialization.
Memory:
If you are working with large datasets, does both your application and the cache have sufficient memory? The memory in your application could be a bottleneck; You should watch your system's activity monitor while performing the operations, and ensure you aren't running out of memory and having your system perform paging. If you find this happening, either consider increasing the RAM, or split the DataTable into smaller datasets as mentioned before.
Latency:
If you are connecting to a Redis server over a network, and not on the same machine, have you checked the latency of the network? You may want to ping your between your application server and the cache server and ensure you actually have a low ping. Particularly if you find caching simple objects is slow.
Redis?
If you are finding there is no way to improve the time to cache and restore, then maybe using Redis isn't a good fit. Perhaps using a static DataTable within the application memory would be more suitable. In other words, by keep the cache in the application memory and then there is no serialization and deserialization to worry about. Of course you may need to be careful about ensuring you have enough memory available to your application to do this. I would however be surprised if you had to choose this option though.
Summary:
Without seeing your dataset or knowledge of the service you are building it's ultimately only generic advise about how to best narrow down the cause of your problem. The key advise is don't use a DataTable if a simpler structure will do, and benchmark each of the operations to determine any bottlenecks.
I hope this helps.
来源：https://stackoverflow.com/questions/20898534/what-is-the-most-time-efficient-way-to-serialize-deserialize-a-datatable-to-from