Metacat

To properly fix this bug, we'll need to add tests for character conversions, to make sure that we aren't introducing regressions with our fixes. Add test documents which break the system, and then afterward add the necessary fixes.

register-dataset.cgi has no testing, and may also be the source of this bug.

Register-dataset.cgi may not have any testing itself, but it is based on Metacat.pm which does have a test suite in the module definition library. So, I think that extending the tests in Metacat.pm should help in covering multiple scripts like register-dataset.cgi that might make use of it for inserting and updating data to metacat.

Using the dev skin to load an XML document with an ampersand encoded as: & kept the character intact. To me, this indicates that Metacat's servlet API is correctly handling the character. If the register-dataset.cgi or Metacat.pm is doing something to encode this additionally, that might account for the double encoding.

Using the Java MetacatClient API to insert a document with & also worked fine (no additional encoding of the & symbol).

The specific example looks to have its origins in delNormalize (sic) within register-dataset.cgi -- the first regex operator replaces '&' with '&' but the last regex operator then replaces '&' with '%26'. I'd recommend removing these functions and using existing modules for encoding/decoding of the XML. There are a couple of options [1], though perhaps just fixing the symptom is good enough for the time being.

1. http://stackoverflow.com/questions/1137790/how-can-i-escape-text-for-an-xml-document-in-perl

I only see delNormalize() being called from the deleteData() function in register-dataset.cgi -- but perhaps similar code is lurking somewhere else?
Is this something you (Shaun) can look into?

I just tried the registry on a test machine running most recent Metacat trunk and the ampersand was correctly encoded for XML (and only once):

<title>Testing & Stuff</title>

Original Bugzilla ID was 3815