Howdy, Stranger!

It looks like you're new here. If you want to get involved, click one of these buttons!

Welcome to the CollectiveAccess support forum! Here the developers and community answer questions related to use of the software. Please include the following information in every new issue posted here:

  1. Version of the software that is used, along with browser and version

  2. If the issue pertains to Providence, Pawtucket or both

  3. What steps you’ve taken to try to resolve the issue

  4. Screenshots demonstrating the issue

  5. The relevant sections of your installation profile or configuration including the codes and settings defined for your local elements.


If your question pertains to data import or export, please also include:

  1. Data sample

  2. Your mapping


Answers may be delayed for posts that do not include sufficient information.

Providence Slowness

We are facing performance issues with Collective Access systems. We are using about 10 Collective Access systems; all of them are quite slow.
The slowness has increased to a point that we are now looking for a tangible solution to this problem. In this regard, we would like to get feedback from other users of Collective Access, whether their Collective Access systems are slow as well or we are the only one facing this problem.

All of our Collective Access systems show slowness in all kinds of operations. From login to search, from viewing a record in editor to saving a record.
Login to Collective Access sometimes takes several minutes, loading an object in object editor takes several seconds, etc. Larger exports of search results (let say pdf of 1500 objects) almost never succeed.

We use Collective Access on Centos 7 machine with MySQL database (installed on the same machine to reduce network overhead for database queries).
Number of records in our systems varies from system to system, for example, number of object type records range from 7000 to 50000.

Collective Access version that we use are: 1.4, 1.7.5 and 1.7.6. We are using Collective Access for a long time, in all versions that we have used until now slowness is a common factor.

If you are a Collective Access user please respond, as your response will help us understand whether this is a common problem among all users or not.

Thanks in advance for your response.

Comments

  • Hi Naeem,

    I've seen that the ACL layer can endorse a large part of the slowness, most of time if it's a question of viewing access to the records. When the ACL is only configured to grant write access, system is faster. I do handle here more than 50 installations, from 2000 to more than 150 000 objects, some with real large pdf collections that are indexed inside the database, french speaking users almost exclusively (some mixed french/dutch in belgium). I've seen that PDF datas indexed from pdfminer can be huge, I mostly rely on pdftotext. For the largest database (2 years old), I've just logged in [2.4518s/94.25M] (footer info) ;

    I do pack a single, dedicated and physical server per database (and maintain it, that's the big part of the challenge). Basic optimizations are tweaking php opcache, redis (often only for Pawtucket), apache mpm workers numbers, mysql innodb settings ; restrict queue processing to single slot, no more than 5 medias, each 3 minutes, and so on. Most of the performances issues are linked to concurrent access on Pawtucket, but with redis, browse facets caching, etc, and limiting apache ressources goes quite well. Those machines use a ligthweight Debian 9 installation, quite all run under PHP 7.2 now.

    CollectiveAccess can show sometimes slow here, but most of times it's a question of resources or a mistake I made at some point. Digging a bit on the installation, restarting apache sometimes gave me back good results.

    Normally, loading a record inside the editor keeps under 2s, even with more than 30 bundles or with customized displays foreach. The login times depends on the dashboard widgets (I have a not-so-good-coded-by-me widget somewhere), rarely more than 2/3s.

    I hope I've given some hints for you, bests,

    Gautier

  • edited November 27

    If any one thing in the request is slow the entire request will be slow. If only certain actions are slow then it points to specific elements within that action being problematic. For example, if certain screens on your object editor take too long to render then it implies one or more elements on the screen are performing poorly for some reason. Sometimes these are things we can improve in the code, sometimes it's due to your system and/or server configuration or the nature of your data.

    It's just about impossible to diagnose a fix for a general "it's slow" complaint. That you're saying that everything is slow, including logins, implies that it's a low level issue. I'm assuming these systems are using varied schemata.

    A few tips to start with:

    1. Run PHP 7.1 or better
    2. Make sure OpCache is enabled and appropriately configured
    3. Optimize your MySQL server (either talk to your DBA about this or Google for tips)
    4. Run current versions of CA, not versions like 1.4 that haven't been updated in forever and don't incorporate performance improvements that have been made over the past several years.
    5. Verify the performance of your file storage. We've seen installs using network file systems where performance grinds to a halt. Both MySQL and PHP can have issues with certain types of file systems that manifest as extremely poor performance rather than outright errors.

    This is just basic stuff. If you profile your servers and the application the root causes can be found. But that takes time, while these basic issues can be investigated quickly.

    There are certainly areas of CA that need to improve performance-wise. The browse in particular does not scale well and needs to be refactored (which is in the works). But it is possible to run systems at scales larger than you are efficiently without over-resourcing. We have containerized ~10 systems (all running current code) onto a Kubernetes cluster with 4 cores and 16 gigs of RAM with good results, for example.

Sign In or Register to comment.