Collision of multipart_id_numbering idnos after migration to new server.

edited February 15 in General Support

After migrating the SQL database to another server (mysqldump + SQL-file import), and updating the code from v1.7.9 to v1.8 (git origin/develop), we suddenly had a serious number of colliding idnos, due to auto-increment of multipart_id_numbering handing out numbers that were already taken.

Reason yet unknown. Configuration was identical.


After spending hours to fix this mess, CA was still handing out wrong numbers, until I updated the "seq" value for that idno-signature in the table "ca_multipart_idno_sequences".

Seth wrote (in December 2019):

"The ca_multipart_idno_sequences table is still maintained but not actually used to generate next in sequence. It'll likely go away in the near future."

https://collectiveaccess.org/support/index.php?p=/discussion/300223/serial-numbering-of-idno-seems-broken&#


Do you have any idea what happened here?

Thanks in advance.

Comments

  • edited February 15

    ca_multipart_idno_sequences still exists for now in part to support identifiers with multiple serial elements. Why it failed in 1.8 when you moved I cannot say based upon what you've reported. Reverting to 1.7 on the new machine and observing if the problem persists or not would tell you if it's configuration-related (despite the assumption no configuration has changed) or 1.8 specific. There are changes in 1.8 numbering vs. 1.7, but they are mainly to do with minor bug fixes and some simplification of configuration. There are no new features or major changes to how they're stored.

    If the multipart_id_numbering.conf is the same, the content of ca_multipart_idno_sequences is the same, and the sortable values in the table for which new numbers were being generated was the same you should not be getting collisions.

    One (unlikely) possibility for the issue you encountered is that you have constructs in your multipart_id_numbering.conf file that are being treated differently by the configuration file parser in 1.8 vs 1.7.9. Character escaping in 1.8 config files is done somewhat differently. These changes will not make a difference in normal use, but it you have unquoted strings with commas or unbalanced quotes in the file it may be stomping values and causing numbers to be treated in unexpected ways that differ from 1.7.x behavior. Just a guess though, but it'd be worth taking a closer look at your multipart_id_numbering.conf file.

    1.8 is obviously not a release yet, but it's seen a fair amount of use and we've not seen any issues like this when moving systems from 1.7.x. I'd be very curious to track down the cause of your issue in case it is 1.8-specific.

  • Thanks for your quick reply!

    About character escaping in "multipart_id_numbering.conf" (Attached. Had to be zipped to be allowed to upload :)):

    Do you mean escaping of special chars or putting quotes around strings?

    The signatures are somewhat like this: "WIFI-DO-1-xxxx" (where "xxxx" is the incrementing SERIAL counter).

    The multipart id numbering actually works as expected. It was just that the numeric index (type=SERIAL) at the end proposed a number that was still within the range of existing idnos. I imagined something like a missing DB index or so, since the migration was done importing the .sql-File created with "mysqldump".


    I've noticed though, that the "seq" indices in the "ca_multipart_idno_sequences" table still contains numbers that are significantly lower than the ones Providence is now suggesting when creating new objects.

    Example:

    "WIFI-MM-1" seq=505, but suggested is "2007"

    The "2007" is actually correct, since the last signature assigned (manually by an operator) was "WIFI-MM-1-2006".


    btw: The SERIAL part of the signature is set to "editable=1" to allow manual adjustments by operators if necessary.

  • edited February 16

    I mean both escaping and quoting as a means of escaping. Looking at your file I don't think this was the issue – unsurprising as it was a long-shot guess.

    Which format or formats in your configuration were issuing already used identifiers after the migration? There are a number of ones with SERIAL elements for object types, as well as one for a collection type. Was it doing this across the board? Or only for specific types of records?

    Also, when you moved servers did your PHP and/or MySQL versions change as well?


    Seth

  • Version changes:

    We switched the http-server from Ubuntu server 16.04 to 18.04, so:

    from PHP 7.0.33-0ubuntu0.16.04.7 to PHP 7.2.24-0ubuntu0.18.04.7

    The database server stayed as-is (therefore no change in MySQL version).


    The issue occurred with objects of a certain sub-type we call "Documents". However, since the migration no other objects have been created, but I've done some spot-checking and the counter for the other objects seems to be fine (now).

  • edited February 17

    Hi Peter,

    So just to confirm: after migration the only object type with the duplicate-new-identifier problem was "Documents"? All others functioned properly?

    How were the objects of this type created? Were they entered by hand or imported? If imported, with what means were they imported?


    Thanks,

    Seth

  • Hi Seth,

    Confirmed: Currently, this only happened with "Documents". So far no other incidents.

    The objects were created manually. No import involved throughout the whole installation.


    Thanks! Peter.

  • edited February 22

    Thank you. That is useful information.

Sign In or Register to comment.