Performance issue with Volley's DiskBasedCache

Question

In my Photo Collage app for Android I\'m using Volley for loading images. I\'m using the DiskBasedCache (included with volley) with 50 mb storage to prevent re-downloading

Answer 1

In the streamToBytes(), first it will new bytes by the cache file length, does your cache file was too large than application maximum heap size ?

private static byte[] streamToBytes(InputStream in, int length) throws IOException {
    byte[] bytes = new byte[length];
    ...
}

public synchronized Entry get(String key) {
    CacheHeader entry = mEntries.get(key);

    File file = getFileForKey(key);
    byte[] data = streamToBytes(..., file.length());
}

If you want to clear the cache, you could keep the DiskBasedCache reference, after clear time's came, use ClearCacheRequest and pass that cache instance in :

File cacheDir = new File(context.getCacheDir(), DEFAULT_CACHE_DIR);
DiskBasedCache cache = new DiskBasedCache(cacheDir);
RequestQueue queue = new RequestQueue(cache, network);
queue.start();

// clear all volley caches.
queue.add(new ClearCacheRequest(cache, null));

this way will clear all caches, so I suggest you use it carefully. of course, you can doing conditional check, just iterating the cacheDir files, estimate which was too large then remove it.

for (File cacheFile : cacheDir.listFiles()) {
    if (cacheFile.isFile() && cacheFile.length() > 10000000) cacheFile.delete();
}

Volley wasn't design as a big data cache solution, it's common request cache, don't storing large data anytime.

------------- Update at 2014-07-17 -------------

In fact, clear all caches is final way, also isn't wise way, we should suppressing these large request use cache when we sure it would be, and if not sure? we still can determine the response data size whether large or not, then call setShouldCache(false) to disable it.

public class TheRequest extends Request {
    @Override
    protected Response<String> parseNetworkResponse(NetworkResponse response) {
        // if response data was too large, disable caching is still time.
        if (response.data.length > 10000) setShouldCache(false);
        ...
    }
}

Answer 2

Yes, the way DiskBasedCache works it needs to open all the files in initialize(). Which is simply.... not a good idea :-(

You need to make a different implementation that doesent open all the files at startup.

Take a copy of DiskBasedCache and change initialize() to

  @Override
  public synchronized void initialize() {
    if (!mRootDirectory.exists()) {
      if (!mRootDirectory.mkdirs()) {
        VolleyLog.e("Unable to create cache dir %s", mRootDirectory.getAbsolutePath());
      }
    }
  }

And change get() so it makes an additional check for if the file exists on the file system, like

  @Override
  public synchronized Entry get(String key) {
    CacheHeader entry = mEntries.get(key);
    File file = getFileForKey(key);
    if (entry == null && !file.exists()) { // EXTRA CHECK
      // if the entry does not exist, return.
      VolleyLog.d("DrVolleyDiskBasedCache miss for " + key);
      return null;
    }
    ...

I use this approach in https://play.google.com/store/apps/details?id=dk.dr.radio and it works fine - its robustness have been tested by ~300000 users :-)

You can download a full version of the file from https://code.google.com/p/dr-radio-android/source/browse/trunk/DRRadiov3/src/dk/dr/radio/net/volley/DrDiskBasedCache.java (you'll have to delete some DR Radio specific stuff)

Answer 3

My initial thought was to use the DiskLruCache written by Jake Wharton by writing a com.android.volley.Cache wrapper over it.

But I finally implemented a singleton pattern for the Volley, combined with the cache creation in an AsyncTask called from the Application context

public static synchronized VolleyClient getInstance(Context context)
{
    if (mInstance == null)
    {
        mInstance = new VolleyClient(context);
    }
    return mInstance;
}

private VolleyClient(Context context)
{
    this.context = context;
    VolleyCacheInitializer volleyCacheInitializer = new VolleyCacheInitializer();
    volleyCacheInitializer.execute();
}

private class VolleyCacheInitializer extends AsyncTask<Void, Void, Boolean>
{
    @Override
    protected Boolean doInBackground(Void... params)
    {
        // Instantiate the cache with 50MB Cache Size
        Cache diskBasedCache = new DiskBasedCache(context.getCacheDir(), 50 * 1024 * 1024);

        // Instantiate the RequestQueue with the cache and network.
        mRequestQueue = new RequestQueue(diskBasedCache, network);

        // Start the queue which calls the DiskBasedCache.initialize()
        mRequestQueue.start();

        return true;
    }

    @Override
    protected void onPostExecute(Boolean aBoolean)
    {
        super.onPostExecute(aBoolean);

        if(aBoolean)
            Log.d(TAG, "Volley request queue initialized");
        else
            Log.d(TAG, "Volley request queue initialization failed");
    }
}

Inside MyApplication class

@Override
public void onCreate()
{
    super.onCreate();

    // Initialize an application level volley request queue
    VolleyClient volleyHttpClient = VolleyClient.getInstance(this);
}