Metacat: Issueshttps://projects.ecoinformatics.org/ecoinfo/https://projects.ecoinformatics.org/ecoinfo/ecoinfo/favicon.ico?14691340362017-12-20T07:00:31ZEcoinformatics Redmine
Redmine Bug #7234 (New): Validate SystemMetadata.checksumAlgorithm in the DataONE API callshttps://projects.ecoinformatics.org/ecoinfo/issues/72342017-12-20T07:00:31ZChris Jonescjones@nceas.ucsb.edu
<p>Bryce pointed out that we have many incorrect <code>checksumAlgorithm</code> strings various MNs. See <a class="external" href="https://github.nceas.ucsb.edu/KNB/arctic-data/issues/283">https://github.nceas.ucsb.edu/KNB/arctic-data/issues/283</a>. The upshot is that <code>SHA-*</code> is the broadly supported syntax.</p>
<p>I checked the strings with:<br /><pre>
package org.dataone.tests;
import java.security.MessageDigest;
import java.security.NoSuchAlgorithmException;
import java.util.ArrayList;
import java.util.List;
public class MessageDigestDTest {
public static void main(String[] args) {
MessageDigest md = null;
List<String> algorithms = new ArrayList<String>();
algorithms.add("MD5");
algorithms.add("MD-5");
algorithms.add("SHA1");
algorithms.add("SHA-1");
algorithms.add("SHA224");
algorithms.add("SHA-224");
algorithms.add("SHA256");
algorithms.add("SHA-256");
algorithms.add("SHA384");
algorithms.add("SHA-384");
algorithms.add("SHA512");
algorithms.add("SHA-512");
for (String algorithm : algorithms) {
try {
md = MessageDigest.getInstance(algorithm);
System.out.println(md.getAlgorithm() + " is recognized.");
} catch (NoSuchAlgorithmException e) {
System.out.println(e.getMessage());
}
}
}
}
</pre></p>
<p>and got:<br /><pre>
MD5 is recognized.
MD-5 MessageDigest not available
SHA1 is recognized.
SHA-1 is recognized.
SHA224 MessageDigest not available
SHA-224 is recognized.
SHA256 MessageDigest not available
SHA-256 is recognized.
SHA384 MessageDigest not available
SHA-384 is recognized.
SHA512 MessageDigest not available
SHA-512 is recognized.
</pre></p>
<p>Change <code>MNodeService</code>, <code>CNodeService</code>, and <code>D1NodeService</code> methods that send or receive <code>SystemMetadata</code> documents and validate the given string with <code>MessageDigest.getInstance(algorithm)</code>. If we get a <code>NoSuchAlgorithm</code> exception, throw an <code>InvalidSystemMetadata</code> exception for the call.</p> Bug #7210 (New): View service duplicates EML Text contenthttps://projects.ecoinformatics.org/ecoinfo/issues/72102017-09-15T00:13:10ZBryce Mecummecum@nceas.ucsb.edu
<p>This abstract<br /><pre><code class="xml syntaxhl">
<span class="nt"><abstract></span>
<span class="nt"><section></span>
<span class="nt"><title></span>Introduction<span class="nt"></title></span>
<span class="nt"><para></span>Between 1958 and 1999, Austin Post led the USGS collection of aerial imagery of North American glaciers. These images are primarily vertical stereo black and white images, although single oblique images, as well as color images have been collected. The glaciers of North America were the subjects, and the digital products made available here serve to document the changes that have occurred to the glaciers over the past 5 decades. The purpose of this project is to preserve the data contained within these film images in a digital format for future analysis of North American glacier change.<span class="nt"></para></span>
<span class="nt"></section></span>
<span class="nt"><section></span>
<span class="nt"><title></span>File Layout<span class="nt"></title></span>
<span class="nt"><para></span>
<span class="nt"><orderedlist></span>
<span class="nt"><listitem></span>
<span class="nt"><para></span>The first level contains an overall data set of image metadata from 1964 - 1997 (nagapData.csv) and an R script (searchData.R) with instructions on how to search and subset the data. fileLayout.pdf shows the file structure and folder contents visually. There are also three kml files with flight path information by decade.<span class="nt"></para></span>
<span class="nt"></listitem></span>
<span class="nt"><listitem></span>
<span class="nt"><para></span>The second level is the year in which the pictures were taken. There are 32 years with images from 1964 – 1997. The majority of these folders are jpegs with notes provided by Austin Post. They also contain a year-specific csv (YYYY.csv) that contains image metadata for the entire year (date, roll numbers, location name, longitude, latitude, altitude, media, and comments). The overall data set (nagapData.csv) is the aggregate of each individual “YYYY.csv” file.<span class="nt"></para></span>
<span class="nt"></listitem></span>
<span class="nt"><listitem></span>
<span class="nt"><para></span>The glacier photos are located at the third level (this level). The folders at this level are distinguished by camera roll number (1, 2, etc.), and image type (thumbnail, jpeg, or tif); some also contain fiducial and oblique image folders. This level primarily contains image files of aerial photos as either thumbnails, jpegs, or tifs. It also includes a csv with image metadata specific to each roll (date, roll numbers, location name, longitude, latitude, altitude, media, and comments), a text file (info.txt) with camera specifications unique to each image, and a text file (histo.txt or matchReport.txt) with color information and scanner specifications unique to each image.<span class="nt"></para></span>
<span class="nt"></listitem></span>
<span class="nt"></orderedlist></span>
<span class="nt"></para></span>
<span class="nt"></section></span>
<span class="nt"></abstract></span>
</code></pre></p>
<p>produces the following HTML:</p>
<pre><code class="xml syntaxhl"><span class="nt"><div</span> <span class="na">class=</span><span class="s">"sectionText"</span><span class="nt">></span>
<span class="nt"><h4</span> <span class="na">class=</span><span class="s">"bold"</span><span class="nt">></span>Introduction<span class="nt"></h4></span>
<span class="nt"><p></span>Between 1958 and 1999, Austin Post led the USGS collection of aerial imagery of North American glaciers. These images are primarily vertical stereo black and white images, although single oblique images, as well as color images have been collected. The glaciers of North America were the subjects, and the digital products made available here serve to document the changes that have occurred to the glaciers over the past 5 decades. The purpose of this project is to preserve the data contained within these film images in a digital format for future analysis of North American glacier change.<span class="nt"></p></span>
<span class="nt"></div></span>
<span class="nt"><div</span> <span class="na">class=</span><span class="s">"sectionText"</span><span class="nt">></span>
<span class="nt"><h4</span> <span class="na">class=</span><span class="s">"bold"</span><span class="nt">></span>File Layout<span class="nt"></h4></span>
<span class="nt"><p></span>The first level contains an overall data set of image metadata from 1964 - 1997 (nagapData.csv) and an R script (searchData.R) with instructions on how to search and subset the data. fileLayout.pdf shows the file structure and folder contents visually. There are also three kml files with flight path information by decade.<span class="nt"></p></span>
<span class="nt"><p></span>The second level is the year in which the pictures were taken. There are 32 years with images from 1964 <span class="ni">&ndash;</span> 1997. The majority of these folders are jpegs with notes provided by Austin Post. They also contain a year-specific csv (YYYY.csv) that contains image metadata for the entire year (date, roll numbers, location name, longitude, latitude, altitude, media, and comments). The overall data set (nagapData.csv) is the aggregate of each individual <span class="ni">&ldquo;</span>YYYY.csv<span class="ni">&rdquo;</span> file.<span class="nt"></p></span>
<span class="nt"><p></span>The glacier photos are located at the third level (this level). The folders at this level are distinguished by camera roll number (1, 2, etc.), and image type (thumbnail, jpeg, or tif); some also contain fiducial and oblique image folders. This level primarily contains image files of aerial photos as either thumbnails, jpegs, or tifs. It also includes a csv with image metadata specific to each roll (date, roll numbers, location name, longitude, latitude, altitude, media, and comments), a text file (info.txt) with camera specifications unique to each image, and a text file (histo.txt or matchReport.txt) with color information and scanner specifications unique to each image.<span class="nt"></p></span>
<span class="nt"><p></span>
The first level contains an overall data set of image metadata from 1964 - 1997 (nagapData.csv) and an R script (searchData.R) with instructions on how to search and subset the data. fileLayout.pdf shows the file structure and folder contents visually. There are also three kml files with flight path information by decade.
The second level is the year in which the pictures were taken. There are 32 years with images from 1964 <span class="ni">&ndash;</span> 1997. The majority of these folders are jpegs with notes provided by Austin Post. They also contain a year-specific csv (YYYY.csv) that contains image metadata for the entire year (date, roll numbers, location name, longitude, latitude, altitude, media, and comments). The overall data set (nagapData.csv) is the aggregate of each individual <span class="ni">&ldquo;</span>YYYY.csv<span class="ni">&rdquo;</span> file.
The glacier photos are located at the third level (this level). The folders at this level are distinguished by camera roll number (1, 2, etc.), and image type (thumbnail, jpeg, or tif); some also contain fiducial and oblique image folders. This level primarily contains image files of aerial photos as either thumbnails, jpegs, or tifs. It also includes a csv with image metadata specific to each roll (date, roll numbers, location name, longitude, latitude, altitude, media, and comments), a text file (info.txt) with camera specifications unique to each image, and a text file (histo.txt or matchReport.txt) with color information and scanner specifications unique to each image.
<span class="nt"></p></span>
<span class="nt"></div></span>
</code></pre>
<p>which you can see duplicates the content in the ordreredlist. The content shouldn't be duplicated.</p> Task #7204 (New): Use the ldaps protocal to connect the ldap serverhttps://projects.ecoinformatics.org/ecoinfo/issues/72042017-07-31T23:41:44ZJing Taotao@nceas.ucsb.edu
<p>Current we use the StartTlsRequest to secure the connection between the Metacat and ldap server. The url for this connection is ldap://ldap.ecoinformatics.org:389/. Nick is thinking to close the port 389 and leave port 636 open, which is used for the ldaps protocol. So we may need to change the url to ldaps and some code as well. We also need to take a look at the ldapweb code as well.</p> Bug #7199 (New): Upgrade postgresql jdbc jar file on Metacathttps://projects.ecoinformatics.org/ecoinfo/issues/71992017-06-02T21:03:49ZJing Taotao@nceas.ucsb.edu
<p>Please see detail on this ticket:<br /><a class="external" href="https://redmine.dataone.org/issues/8104">https://redmine.dataone.org/issues/8104</a></p> Bug #7181 (New): Verify completeness of unit test MetacatRdfXmlSubprocessorTesthttps://projects.ecoinformatics.org/ecoinfo/issues/71812017-04-11T23:41:43ZPeter Slaughterslaughter@nceas.ucsb.edu
<p>Verify that all prov relationships that are indexed via src/main/resources/application-context-prov-base.xml are inspected by the unit test MetacatRdfXmlSubprocessorTest.java which reads src/test/resources/rdfxml-example.xml.</p> Bug #7164 (New): View service rendering EML project abstract incorrectlyhttps://projects.ecoinformatics.org/ecoinfo/issues/71642016-12-01T21:38:40ZBryce Mecummecum@nceas.ucsb.edu
<p>The view service's XSLT for rendering EML is producing output that doesn't look quite right. See attached screenshot. A quick investigating revealed it's just missing a wrapping div.</p> Story #7148 (New): Upgrade Solr Serverhttps://projects.ecoinformatics.org/ecoinfo/issues/71482016-10-19T23:08:05ZJing Taotao@nceas.ucsb.edu
<p>Currently, the solr server version on Metacat is 3.6.2. But the newest version is 6.*. We need to upgrade it in the near future.</p> Bug #7092 (New): Upgrade xerces and xalan libhttps://projects.ecoinformatics.org/ecoinfo/issues/70922016-08-25T18:01:37ZJing Taotao@nceas.ucsb.edu
<p>Currently we are using xerces and xalan 2.7 version which came out in 2005. We need to upgraded it.</p> Story #6437 (New): Upgrade to SOLR 4 or 5https://projects.ecoinformatics.org/ecoinfo/issues/64372014-03-03T14:52:16Zben leinfelderleinfelder@nceas.ucsb.edu
We'd like to make use of new key features in SOLR 4:
<ul>
<li>spatial indexing/filters</li>
<li>geohash (for heatmaps/displaying all spatial results at once)</li>
<li>multiple nested facets</li>
</ul> Bug #4840 (New): When registering, U.S. State/Territory of owner is also applied to contacthttps://projects.ecoinformatics.org/ecoinfo/issues/48402010-02-24T23:50:25ZJim Regetzregetz@nceas.ucsb.edu
<p>When registering a new data set, dropdown menus for "U.S. State or Territory" appear both in the Principal Data Set Owner section and in the Data Set Contact section. However, whatever value is submitted under Owner is evidently getting applied to Contact as well, and the value (if any) selected under Contact is ignored. This is true even if no state was selected under Owner, in which case no state is recorded for Contact.</p>
<p>This was observed both for the ESA and NCEAS registries.</p> Bug #4609 (New): when viewed with saved credentials, KNB homepage shows blank entrieshttps://projects.ecoinformatics.org/ecoinfo/issues/46092009-12-09T00:23:12ZOliver Soongsoong@nceas.ucsb.edu
<p>I sign into the KNB, then close the window (but not the browser). The cookie persists, so when I go back to the KNB home page, it picks up the saved credentials from earlier. However, the login space shows no entries (blank username and password, -- choose one -- for the organization), so I can't be sure who I'm logged in as.</p>
<p>This also happens with dev.nceas.ucsb.edu.</p> Bug #4283 (New): use HTMLUnit to do unit / integration tests on pageshttps://projects.ecoinformatics.org/ecoinfo/issues/42832009-08-06T22:12:31ZShaun Walbridgewalbridge@nceas.ucsb.edu
<p>A number of issues we've had with both the Registry, skins display, and administrative configuration utility could be caught with some unit tests on the pages themselves. This product is fairly robust and is Java, which should make it relatively straightforward to add to our existing testing routines.</p> Bug #3592 (New): Update sitemap generation and registrationhttps://projects.ecoinformatics.org/ecoinfo/issues/35922008-11-04T19:44:10ZMichael Daigledaigle@nceas.ucsb.edu
<p>Update web search sitemap functionality so that:</p>
<p>-- the sitemap generation is done at a scheduled time.<br />-- the sitemaps will be registered if they have changed.</p> Bug #2228 (New): Use method other than metadata string filters for determining the organizations ...https://projects.ecoinformatics.org/ecoinfo/issues/22282005-10-11T23:04:46ZWill Tyburczytyburczy@nceas.ucsb.edu
<p>Right now when data is registered through an organization's skin, the metadata<br />for the entry is made to include a specific string, such as "UC Natural Reserve<br />System". When listing all the data packages from that organization, the current<br />method is to use that string as a filter. Unfortunately, if a data package<br />affiliated with an organization (such as the NRS) is submitted via a different<br />registry (such as ESA), the data package won't show up in the list of that<br />organization's data. Thus, a different approach is needed.</p>
<p>From Matt's email:</p>
<p>"The right way to do this would be to decouple the metadata entry from <br />the filter that controls whether a given entry shows up in the registry. <br /> We've been experimenting with some ways to do this through 'semantic <br />annotations' that are maintained separately from the rest of the <br />metadata. This would allow us to retroactively label data packages with <br />annotations that could control which skin they show up in. It still <br />doesn't solve the maintenance problem (for each new registry skin that <br />we create, we need to annotate all past data packages that we want to <br />show up in the skin). I'm still not sure how to deal with that issue."</p> Bug #1122 (In Progress): XML document is a string in HttpServerletRequest objecthttps://projects.ecoinformatics.org/ecoinfo/issues/11222003-07-30T00:34:44ZJing Taotao@nceas.ucsb.edu
<p>When morpho or loadxml.html in metacat upload a xml document to metacat, it <br />allways stores the xml document in a parameter named "doctext". Servelet will<br />store this parameter name and value (xml documentment itself) into a hashtable.<br />And xml document will be stored as a String in memory. So if xml document is <br />big ( huge inline data in eml2 document), it will cause memory problem.</p>
<p>The option we can choose is, using a stream (or reader) connecting client side and servlet <br />directly, rather than transfer the xml document to a string. Our SAX parser wil<br />parse the stream.</p>