Howdy, Stranger!

It looks like you're new here. If you want to get involved, click one of these buttons!

Welcome to the CollectiveAccess support forum! Here the developers and community answer questions related to use of the software. Please include the following information in every new issue posted here:

  1. Version of the software that is used, along with browser and version

  2. If the issue pertains to Providence, Pawtucket or both

  3. What steps you’ve taken to try to resolve the issue

  4. Screenshots demonstrating the issue

  5. The relevant sections of your installation profile or configuration including the codes and settings defined for your local elements.

If your question pertains to data import or export, please also include:

  1. Data sample

  2. Your mapping

Answers may be delayed for posts that do not include sufficient information.

Importer Setting Rule seems to be ignored

edited October 2017 in Troubleshooting
I am stumped as to why the Data Importer Setting “existingRecordPolicy” is being ignored. Regardless of the option, the next data import simply creates a new record. None of the “merge”, “skip”, or “overwrite” options have any apparent effect. Any option defined just results in a new record being created. This even results with duplicate idno records being created, which I didn't think was allowed. Below is my import mapping file and sample data (which I have been using to attempt to merge, replace, etc).Providence 1.7.5


  • I need to make a partial correction. The options using only preferred_labels do work, but not those using "idno".
    What is wrong? I seem to be correctly importing the "idno" to ca_objects.idno. I am able to search for this idno and all records using this duplicate id are listed. I am even warned that the same id is used multiple times. If the importer map sets "existingRecordPolicy" to "skip_on_idno", shouldn't the import avoid creating duplicate records with this same id?
  • Your mapping & data works perfectly for me with either skip_on_idno or merge_on_idno. (I'm running the lastest from Github/develop 1.7.6 - Schema revision 147). Are you sure there isn't a typo in the existing record policy setting? What does your log say? Do you want to try updating?
  • Just tried it in a 1.7.5 environment and it's working there too.
  • edited October 2017
    The only type of mapping file I am able to use is "XLSX'. A "CSV" file type uploads but doesn't get recognized as the file type in the "import data" screen. There is always a possibility of a problem with the XLS mapping but I have attempted a number of different versions. Another possibility is my use of Open Office to create XLS mapping files. XLSX is not a save option with Open Office. If 1.7.6 is good to go, I can have it installed by Friday.
    By the way, I have my CA log level set to debug. I see no complaints. The idno doesn't seem to get recognized and another record is simply added. CaUtils also imports data, ignoring idno.
  • No need to update to 1.7.6 because I can confirm it's not a bug and is working in 1.7.5. Sounds like the issue lies with your hosting environment and/or mapping file type.
  • edited October 2017
    Success! I discovered the MYSQL user defined in my setup.php did not have full privileges. The "skip_on_idno" problem went away once I set FULL PRIVILEGES access for this user. Let this be a warning to other novices. My installation largely worked without full privileges and I saw no clues that anything was setup wrong with the database user defined in setup.php.

    Another issue I am troubleshooting is my import mapping and data import editor: Open Office 4.1.3. I cannot write XLSX files, so I am working with XLS formatted files. It seems to work okay until I try and delete rows and get unexpected results. For example, suppose I create a data sheet to import and enter data on 20 rows. I then decide to delete 10 rows, save the file, and then use it to import the 10 remaining records. CA will import my 10 intended records, but then will also import the 10 deleted records with BLANK content. The spreadsheet appears only to contain 10 rows, but something of the deleted rows seems to survive in the spreadsheet and still gets imported to CA. Beware, Open Office users!
  • Try copying and pasting the 10 intended rows into a new document. That should resolve the blanks issue.
Sign In or Register to comment.