Howdy, Stranger!

It looks like you're new here. If you want to get involved, click one of these buttons!

Welcome to the CollectiveAccess support forum! Here the developers and community answer questions related to use of the software. Please include the following information in every new issue posted here:

  1. Version of the software that is used, along with browser and version

  2. If the issue pertains to Providence, Pawtucket or both

  3. What steps you’ve taken to try to resolve the issue

  4. Screenshots demonstrating the issue

  5. The relevant sections of your installation profile or configuration including the codes and settings defined for your local elements.


If your question pertains to data import or export, please also include:

  1. Data sample

  2. Your mapping


Answers may be delayed for posts that do not include sufficient information.

pawtucket tmp

Hi!

How does the pawtucket/app/tmp system work? Is it really normal to have many many thousands of directories generated every day? Our installation of Pawtucket stopped working a couple of months ago due to the enormous amount of directories and files in the tmp directory. I manually deleted all the directories (the system couldn't build the list) and it started working again. Now we empty the collectiveaccessCache directory once a day through Crontab, but sometimes Pawtucket generate so many directories int the tmp directory that a rm -r command can't delete them.

What is best practice here?

Regards

Henrik

Comments

  • We use a modular cache system that support several back-ends, includes the file system (the default), redis and memcached. The file caching system is taken from Doctrine, and while it's simple to implement it does indeed generate a ton of files. The file back-end is high on my list of things that need to be replaced.

    The best practice at this time, especially for sites that see a lot of traffic, is to use redis for caching. It's packaged for most, if not all, current Linux distributions and Mac OS, and generally just works. Once it's installed and running you just change the cache settings in setup.php and you should be good to go.

    Seth
  • There's now an experimental branch on GitHub (dev/recache) with a different caching layer, that hopefully performs better than Doctrine when used with the file system. As with Doctrine it also supports REDIS, Memcached and APC. There's still a bit more work to do, then we'll put it on a few of our real world systems and see how well it works in practice.


  • Ok!

    I'll check if our Ubuntu 16.4 (TLS) installation has redis installed and give it a go.

    /Henrik
  • FWIW, the new branch now supports sqlite as well. I'm going to test that this week and see how well it performs. In principal, sqlite should be almost as easy as the file cache to get running but without the performance issues on busy sites.
  • edited November 2018

    Seth: We are having a similar issue of cache files not deleting and redis as the cache resolves it on my local instance--thankyou! The production site is hosted on a server that requires a socket connection. Is it possible to configure using a socket instead of using hostname and port? Thanks! BB

  • We had a similar problem after one year of using Pawtucket2 productively – Pawtucket stopped working, the browser stayed on a white, empty page. After I deleted the tmp folder everything worked again. Two days later the same happened, so I’m using a cronjob to delte the tmp folder every night. This is a workaround until I figure out how to change the chacing backend to redis. This is why I’m here – is there any info on how to configure redis? I installed it on our server and updated setup.php, but then Pawtucket stays white immediately. Thanks for any help & guidance!

  • To use redis you'll need to install the redis server package. This is a standard package in many (most?) Linux distributions these days. If you're using CentOS/RedHat it's as simple as typing yum install redis

    Next you'll have to install the php-redis extension. This is also usually packaged these days.

    Finally, in your setup.php file you'll need to set CA_CACHE_BACKEND to "redis"
    If you're going to run the same redis for both front and back-end installations, then be sure to set CA_REDIS_DB to 0 for one and 1 for the other. (Most redis servers are configured out of the box to allow 16 separate "databases", numbered 0 through 15; this is controlled by the "databases" directive in redis.conf)

    If you're running redis on a different machine or non-standard port you may also want to set CA_REDIS_HOST and/or CA_REDIS_PORT.

    That's about all there is to it.

    File caching is certainly simpler, but all of the implementations out there (Doctrine, Stash, etc.) are based upon making thousands upon thousands of tiny files, which doesn't scale wonderfully. When the caches fills cleanup of those files unavoidably grinds things to a halt, and there's nothing we can do about it. As an in-memory cache redis avoids these problems, but it presents its own challenges for busy sites when memory where memory may become exhausted. You may want to run "flushall" on your redis server periodically. Also look carefully at how "maxmemory" and "maxmemory-policy" are set in redis.conf. The default settings for these will cause your system to grind to a halt when memory is exhausted. This may not be a bad thing, but it's worth considering how you want to handle these sort of situations.

  • Hi Seth, thanks! I did as you described, checked that redis is working (redis-cli ping) but my Pawtucket install stays white (no errors) when I change the cache backend to redis.

  • Are there errors in the log? Are you sure you have the php-redis extension installed?

  • I guess I simply forgot to restart apache (httpd) after installing redis

  • That could do it :-)

Sign In or Register to comment.