Ecoinformatics Redmine: Issueshttps://projects.ecoinformatics.org/ecoinfo/https://projects.ecoinformatics.org/ecoinfo/ecoinfo/favicon.ico?14691340362017-09-15T00:13:10ZEcoinformatics Redmine
Redmine Metacat - Bug #7210 (New): View service duplicates EML Text contenthttps://projects.ecoinformatics.org/ecoinfo/issues/72102017-09-15T00:13:10ZBryce Mecummecum@nceas.ucsb.edu
<p>This abstract<br /><pre><code class="xml syntaxhl">
<span class="nt"><abstract></span>
<span class="nt"><section></span>
<span class="nt"><title></span>Introduction<span class="nt"></title></span>
<span class="nt"><para></span>Between 1958 and 1999, Austin Post led the USGS collection of aerial imagery of North American glaciers. These images are primarily vertical stereo black and white images, although single oblique images, as well as color images have been collected. The glaciers of North America were the subjects, and the digital products made available here serve to document the changes that have occurred to the glaciers over the past 5 decades. The purpose of this project is to preserve the data contained within these film images in a digital format for future analysis of North American glacier change.<span class="nt"></para></span>
<span class="nt"></section></span>
<span class="nt"><section></span>
<span class="nt"><title></span>File Layout<span class="nt"></title></span>
<span class="nt"><para></span>
<span class="nt"><orderedlist></span>
<span class="nt"><listitem></span>
<span class="nt"><para></span>The first level contains an overall data set of image metadata from 1964 - 1997 (nagapData.csv) and an R script (searchData.R) with instructions on how to search and subset the data. fileLayout.pdf shows the file structure and folder contents visually. There are also three kml files with flight path information by decade.<span class="nt"></para></span>
<span class="nt"></listitem></span>
<span class="nt"><listitem></span>
<span class="nt"><para></span>The second level is the year in which the pictures were taken. There are 32 years with images from 1964 – 1997. The majority of these folders are jpegs with notes provided by Austin Post. They also contain a year-specific csv (YYYY.csv) that contains image metadata for the entire year (date, roll numbers, location name, longitude, latitude, altitude, media, and comments). The overall data set (nagapData.csv) is the aggregate of each individual “YYYY.csv” file.<span class="nt"></para></span>
<span class="nt"></listitem></span>
<span class="nt"><listitem></span>
<span class="nt"><para></span>The glacier photos are located at the third level (this level). The folders at this level are distinguished by camera roll number (1, 2, etc.), and image type (thumbnail, jpeg, or tif); some also contain fiducial and oblique image folders. This level primarily contains image files of aerial photos as either thumbnails, jpegs, or tifs. It also includes a csv with image metadata specific to each roll (date, roll numbers, location name, longitude, latitude, altitude, media, and comments), a text file (info.txt) with camera specifications unique to each image, and a text file (histo.txt or matchReport.txt) with color information and scanner specifications unique to each image.<span class="nt"></para></span>
<span class="nt"></listitem></span>
<span class="nt"></orderedlist></span>
<span class="nt"></para></span>
<span class="nt"></section></span>
<span class="nt"></abstract></span>
</code></pre></p>
<p>produces the following HTML:</p>
<pre><code class="xml syntaxhl"><span class="nt"><div</span> <span class="na">class=</span><span class="s">"sectionText"</span><span class="nt">></span>
<span class="nt"><h4</span> <span class="na">class=</span><span class="s">"bold"</span><span class="nt">></span>Introduction<span class="nt"></h4></span>
<span class="nt"><p></span>Between 1958 and 1999, Austin Post led the USGS collection of aerial imagery of North American glaciers. These images are primarily vertical stereo black and white images, although single oblique images, as well as color images have been collected. The glaciers of North America were the subjects, and the digital products made available here serve to document the changes that have occurred to the glaciers over the past 5 decades. The purpose of this project is to preserve the data contained within these film images in a digital format for future analysis of North American glacier change.<span class="nt"></p></span>
<span class="nt"></div></span>
<span class="nt"><div</span> <span class="na">class=</span><span class="s">"sectionText"</span><span class="nt">></span>
<span class="nt"><h4</span> <span class="na">class=</span><span class="s">"bold"</span><span class="nt">></span>File Layout<span class="nt"></h4></span>
<span class="nt"><p></span>The first level contains an overall data set of image metadata from 1964 - 1997 (nagapData.csv) and an R script (searchData.R) with instructions on how to search and subset the data. fileLayout.pdf shows the file structure and folder contents visually. There are also three kml files with flight path information by decade.<span class="nt"></p></span>
<span class="nt"><p></span>The second level is the year in which the pictures were taken. There are 32 years with images from 1964 <span class="ni">&ndash;</span> 1997. The majority of these folders are jpegs with notes provided by Austin Post. They also contain a year-specific csv (YYYY.csv) that contains image metadata for the entire year (date, roll numbers, location name, longitude, latitude, altitude, media, and comments). The overall data set (nagapData.csv) is the aggregate of each individual <span class="ni">&ldquo;</span>YYYY.csv<span class="ni">&rdquo;</span> file.<span class="nt"></p></span>
<span class="nt"><p></span>The glacier photos are located at the third level (this level). The folders at this level are distinguished by camera roll number (1, 2, etc.), and image type (thumbnail, jpeg, or tif); some also contain fiducial and oblique image folders. This level primarily contains image files of aerial photos as either thumbnails, jpegs, or tifs. It also includes a csv with image metadata specific to each roll (date, roll numbers, location name, longitude, latitude, altitude, media, and comments), a text file (info.txt) with camera specifications unique to each image, and a text file (histo.txt or matchReport.txt) with color information and scanner specifications unique to each image.<span class="nt"></p></span>
<span class="nt"><p></span>
The first level contains an overall data set of image metadata from 1964 - 1997 (nagapData.csv) and an R script (searchData.R) with instructions on how to search and subset the data. fileLayout.pdf shows the file structure and folder contents visually. There are also three kml files with flight path information by decade.
The second level is the year in which the pictures were taken. There are 32 years with images from 1964 <span class="ni">&ndash;</span> 1997. The majority of these folders are jpegs with notes provided by Austin Post. They also contain a year-specific csv (YYYY.csv) that contains image metadata for the entire year (date, roll numbers, location name, longitude, latitude, altitude, media, and comments). The overall data set (nagapData.csv) is the aggregate of each individual <span class="ni">&ldquo;</span>YYYY.csv<span class="ni">&rdquo;</span> file.
The glacier photos are located at the third level (this level). The folders at this level are distinguished by camera roll number (1, 2, etc.), and image type (thumbnail, jpeg, or tif); some also contain fiducial and oblique image folders. This level primarily contains image files of aerial photos as either thumbnails, jpegs, or tifs. It also includes a csv with image metadata specific to each roll (date, roll numbers, location name, longitude, latitude, altitude, media, and comments), a text file (info.txt) with camera specifications unique to each image, and a text file (histo.txt or matchReport.txt) with color information and scanner specifications unique to each image.
<span class="nt"></p></span>
<span class="nt"></div></span>
</code></pre>
<p>which you can see duplicates the content in the ordreredlist. The content shouldn't be duplicated.</p> Metacat - Bug #7182 (New): Allow partial package downloads when some of the objects are private https://projects.ecoinformatics.org/ecoinfo/issues/71822017-04-13T15:10:57ZLauren Walkerwalker@nceas.ucsb.edu
<p>When you try to download a package that has at least one private object, you get a 401 - Unauthorized response. When I am authorized to read at least one object in a package, I would expect to still be able to download the .zip package with those objects.</p>
<p>It's difficult for MetacatUI to tell when the "Download All" will fail, since it would need to check the /isAuthorized/{pid}?action=read result for every single object in the package, which can sometimes be >100. So right now we have an issue where users are getting a failed package download.</p> Metacat - Bug #7178 (New): MNodeService.getPackage() takes too long for large packageshttps://projects.ecoinformatics.org/ecoinfo/issues/71782017-03-29T14:22:46ZChris Jonescjones@nceas.ucsb.edu
<p>When users click on the <code>Download All</code> button in MetacatUI, we call <code>MN.getPackage()</code> to zip up the members, create HTML and PDF metadata, etc. For very large packages, the packaging time is far too long for a decent user experience. To address this, we may need to provide some sort of progress API call that allows the client to get estimated packaging time and provide a progress bar for the user. Also, <code>getPackage()</code> uses <code>File.createTempFile()</code> to copy contents into a single directory tree for zipping (really BagIt bagging). This doesn't scale well for large packages (GBs). We can explore a few strategies to mitigate this. One that comes to mind is using hard symbolic links to the original data files in the directory tree rather than copying them. This needs some thought, but ultimately we need to speed up the packaging process for large packages.</p> Metacat - Bug #7164 (New): View service rendering EML project abstract incorrectlyhttps://projects.ecoinformatics.org/ecoinfo/issues/71642016-12-01T21:38:40ZBryce Mecummecum@nceas.ucsb.edu
<p>The view service's XSLT for rendering EML is producing output that doesn't look quite right. See attached screenshot. A quick investigating revealed it's just missing a wrapping div.</p> Metacat - Bug #6959 (New): http response charset not includedhttps://projects.ecoinformatics.org/ecoinfo/issues/69592016-02-11T00:29:11ZPeter Slaughterslaughter@nceas.ucsb.edu
<p>Metacat doesn't include the content type charset in the http response headers:</p>
<p>curl -o test.dat -v "https://knb.ecoinformatics.org/knb/d1/mn/v2/object/solson.25.5" returns</p>
<p>Content-Type: text/csv</p>
<p>but should be something like:</p>
<p>Content-Type: text/xml;charset=UTF-8</p>
<p>so that the client doesn't have to assume the charset</p> Metacat - Feature #6848 (New): Add link to send another verification emailhttps://projects.ecoinformatics.org/ecoinfo/issues/68482015-09-30T16:29:26ZLauren Walkerwalker@nceas.ucsb.edu
<p>This has been something suggested by a few users when they don't get their original verification email</p> Metacat - Support #6793 (In Progress): Update DOIs from KNB to redirect to view servicehttps://projects.ecoinformatics.org/ecoinfo/issues/67932015-07-02T19:00:00ZMatt Jonesjones@nceas.ucsb.edu
<p>In <a class="issue tracker-2 status-3 priority-2 priority-default closed" title="Feature: Create a new service to update DOI pointers (Resolved)" href="https://projects.ecoinformatics.org/ecoinfo/issues/6530">#6530</a> and <a class="issue tracker-5 status-5 priority-2 priority-default closed" title="Story: Keep DOI registrations current and resolvable (Closed)" href="https://projects.ecoinformatics.org/ecoinfo/issues/6440">#6440</a>, we added features to update DOI registrations, but we still have many originally assigned DOIs that redirect to the raw EML document rather than our landing page for a data set. We need to fix all of the /AA/ DOI registrations in the KNB and ensure they point to the right View service page. For DOIs for metadata, that would be the associated /view url for that DOI. For data files and resource maps, its to the view for the associated metadata. E.g.,</p>
<ul>
<li>Metadata doi:10.5063/AA/nceas.227.15 should redirect to <a class="external" href="https://knb.ecoinformatics.org/#view/doi:10.5063/AA/nceas.227.15">https://knb.ecoinformatics.org/#view/doi:10.5063/AA/nceas.227.15</a></li>
<li>Data doi:10.5063/AA/wtyburczy.30.1 should redirect to the same metadata file <a class="external" href="https://knb.ecoinformatics.org/#view/doi:10.5063/AA/nceas.227.15">https://knb.ecoinformatics.org/#view/doi:10.5063/AA/nceas.227.15</a></li>
</ul>
<p>Also, when a user updates metadata for a package (but doesn't change the data), the DOI redirect for the data will need to be updated to point to the new metadata. Let's verify that this is happening automatically in Metacat.</p> Morpho - Bug #6702 (New): Import metadata indexing optionhttps://projects.ecoinformatics.org/ecoinfo/issues/67022015-03-27T21:59:28ZJessica Couturecouture@nceas.ucsb.edu
<p>When metadata is added to a package from an external file, Morpho creates a new identifier for it rather than incrementing the existing identifier. It would be great to have the option to either add the metadata to the existing series or create a new identifier when the file is brought in.</p> Morpho - Bug #6589 (New): common name in taxonomic coveragehttps://projects.ecoinformatics.org/ecoinfo/issues/65892014-08-28T20:02:32ZJessica Couturecouture@nceas.ucsb.edu
<p>When inputting taxonomic coverage manually there is an option to put common name but when assigning columns of names from a table, only the scientific classes are available. It would be helpful to be able to add common names from a table too.</p> Morpho - Bug #6045 (In Progress): Morpho not correctly handling Chinese charactershttps://projects.ecoinformatics.org/ecoinfo/issues/60452013-08-05T17:22:53Zben leinfelderleinfelder@nceas.ucsb.edu
<p>Hi Ben,</p>
<p>There is a Chinese display problem in Matacat 2.0.6 when the data package uploading from Morpho 1.10.0.<br />My colleague, Chen-Tsai Chiu, modified the XMLUtil.java(MORPHO_1_10_0/src/edu/ucsb/nceas/morpho/util/).<br />He add codes(red part) to the XMLUtil.jave as following.</p>
<pre>
boolean escape = false;
if (!escape) {
// do not do escape for true UTF-8 encoding
//str.append(ch);
str.append("&#");
str.append(Integer.toString(ch));
str.append(';');
} else {
// escape the special characters
// see http://bugzilla.ecoinformatics.org/show_bug.cgi?id=5238
str.append("&#");
str.append(Integer.toString(ch));
str.append(';');
}
</pre>
<p>We can't make sure the way is correct but it works. <br />Does it happen in other language?</p>
<p>Thank you.</p>
<p>Meei-ru</p> MetacatUI - Bug #6044 (In Progress): Enable sub-text searching in Solr querieshttps://projects.ecoinformatics.org/ecoinfo/issues/60442013-08-01T23:45:32ZChris Jonescjones@nceas.ucsb.edu
<p>When submitting a search to the Solr index, we are only getting hits for whole-word searches. We want to be able to return results for fragments of text, like, a search for 'ocean' would also return hits of 'oceanographic'.</p> Morpho - Bug #6022 (In Progress): Morpho error on government computer with Windows 7https://projects.ecoinformatics.org/ecoinfo/issues/60222013-07-03T19:58:00ZSarah Clarksclark@nceas.ucsb.edu
<p>"Hi Sarah,<br />I am trying to install Morpho on my government computer with Windows 7 and am having some difficulty. I downloaded the file, started the installer, it said it was installed correctly and all seems well, but when I open it, it gives me a "Failed to create config directory' and closes."</p> Morpho - Bug #6010 (New): Windows 8 installation errorhttps://projects.ecoinformatics.org/ecoinfo/issues/60102013-07-01T16:19:59ZSarah Clarksclark@nceas.ucsb.edu
<p>A user trying to install Morpho in Windows 8 received a configuration error, Matt described it as an issue with write access to the installation directory.</p> Metacat - Bug #4886 (New): "DatabaseService is already registered" error after configuring Metacathttps://projects.ecoinformatics.org/ecoinfo/issues/48862010-03-17T20:33:20ZJing Taotao@nceas.ucsb.edu
<p>I installed Metacat from 1.9.2 branch and configured it successfully. Then I typed<br />the url on the browser:<br /><a class="external" href="http://chico1.dyndns.org/knb/metacat?action=read&qformat=knb&docid=tao.14547">http://chico1.dyndns.org/knb/metacat?action=read&qformat=knb&docid=tao.14547</a></p>
<p>I got the error:<br />javax.servlet.ServletException: Service problem while intializing MetaCat Servlet: ServiceService.registerService - Service: DatabaseService is already registered. Use ServiceService.reregister() to replace the service.<br /> edu.ucsb.nceas.metacat.MetaCatServlet.initSecondHalf(MetaCatServlet.java:490)<br /> edu.ucsb.nceas.metacat.MetaCatServlet.handleGetOrPost(MetaCatServlet.java:696)<br /> edu.ucsb.nceas.metacat.MetaCatServlet.doGet(MetaCatServlet.java:519)<br /> javax.servlet.http.HttpServlet.service(HttpServlet.java:690)<br /> javax.servlet.http.HttpServlet.service(HttpServlet.java:803)<br /> org.vfny.geoserver.filters.SetCharacterEncodingFilter.doFilter(SetCharacterEncodingFilter.java:122)</p>
<p>I had to restart tomcat and error was gone.</p>
<p>Paul Were sent an email to metacat-dev describing that he got the same error:</p>
<p>I installed metacat 192RC4 and configured it ok.</p>
<p>I am now attempting run the registry.</p>
<p>I get the Registry login page but whenever I try to log in it fails. I get the following error.</p>
<p>knb 20100316-11:49:55: [ERROR]: MetaCatServlet.initSecondHalf - Service problem while initializing MetaCat Servlet: ServiceService.registerService - Service:<br />DatabaseService is already registered. Use ServiceService.reregister() to replace the service. [edu.ucsb.nceas.metacat.MetaCatServlet]</p>
<p>I am able to login into the admin page with no error as</p>
<p>uid=pwere,o=unaffiliated,dc=ecoinformatics,dc=org</p>
<p>and that is the same login I am using for the registry.</p>
<p>Jing's comment: i guess the "registry" in Paul's email means metacat.</p> Metacat - Bug #1324 (New): A Commit or Save Work button that wrote intermediate results to the me...https://projects.ecoinformatics.org/ecoinfo/issues/13242004-02-06T03:38:14ZSaurabh Gargsgarg@nceas.ucsb.edu
<p>From Ricks comments:<br />Entering metadata using the Data Repository screen can take 10 to 30 minutes or <br />longer; sometimes Internet connections can fail or browsers crash, which leads <br />to loss of data. It would be reassuring and productivity-enhancing if the <br />screen had a ‘Commit’ or ‘Save Work’ button that wrote intermediate results to <br />the metadata. I realize that this can be done by saving and then re-opening the <br />(existing) data package for further entry; but a ‘one-button’ solution would be <br />even better.</p>