project-cdsware-announce@cern.ch archives


[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

CDS Invenio v0.99.0 is released


  • From: Tibor Simko <tibor.simko@xxxxxxx>
  • Subject: CDS Invenio v0.99.0 is released
  • Date: Thu, 27 Mar 2008 12:27:30 +0100

--------------------------------------------------------------------
CDS Invenio v0.99.0 is released
March 27, 2008
http://cdsware.cern.ch/invenio/news.html
--------------------------------------------------------------------

CDS Invenio v0.99.0 was released on March 27, 2008.

What's new:
-----------

 *) new Invenio configuration language, new inveniocfg configuration
    tool permitting more runtime changes and enabling separate local
    customizations (MiscUtil)

 *) phased out WML dependency everywhere (all modules)

 *) new common RSS cache implementation (WebSearch)

 *) improved access control to the detailed record pages (WebSearch)

 *) when searching non-existing collections, do not revert to
    searching in public Home anymore (WebSearch)

 *) strict calculation of number of hits per multiple collections
    (WebSearch)

 *) propagate properly language environment in browse pages, thanks to
    Ferran Jorba (WebSearch)

 *) search results sorting made accentless, thanks to Ferran Jorba
    (WebSearch)

 *) new OpenURL interface (WebSearch)

 *) added new search engine API argument to limit searches to record
    creation/modification dates and times instead of hitherto creation
    dates only (WebSearch)

 *) do not allow HTTP POST method for searches to prevent hidden
    mining (WebSearch)

 *) added alert and RSS teaser for search engine queries (WebSearch)

 *) new optimized index structure for fast integer bit vector
    operations, leading to significant indexing time improvements
    (MiscUtil, BibIndex, WebSearch)

 *) new tab-based organisation of detailed record pages, with new URL
    schema (/record/1/usage) and related CSS changes (BibFormat,
    MiscUtil, WebComment, WebSearch, WebStyle, WebSubmit)

 *) phased out old PHP based code; migration to Python-based output
    formats recommended (BibFormat, WebSubmit)

 *) new configurability to show/hide specific output formats for
    specific collections (BibFormat, WebSearch)

 *) new configurability to have specific stemming settings for
    specific indexes (BibIndex, WebSearch)

 *) optional removal of LaTeX markup for indexer (BibIndex, WebSearch)

 *) performance optimization for webcoll and optional arguments to
    refresh only parts of collection cache (WebSearch)

 *) optional verbosity argument propagation to the output formatter
    (BibFormat, WebSearch)

 *) new convenient reindex option to the indexer (BibIndex)

 *) fixed problem with indexing of some lengthy UTF-8 accented names,
    thanks to Theodoros Theodoropoulos for reporting the problem
    (BibIndex)

 *) fixed full-text indexing of HTML pages (BibIndex)

 *) new Stemmer module dependency, fixes issues on 64-bit systems
    (BibIndex)

 *) fixed download history graph display (BibRank)

 *) improved citation ranking and history graphs, introduced
    self-citation distinction, added new demo records (BibRank)

 *) fixed range redefinition and output message printing problems in
    the ranking indexer, thanks to Mike Marino (BibRank)

 *) new XSLT output formatter support; phased out old BFX formats
    (BibFormat)

 *) I18N output messages are now translated in the output formatter
    templates (BibFormat)

 *) formats fixed to allow multiple author affiliations (BibFormat)

 *) improved speed of the record output reformatter in case of large
    sets (BibFormat)

 *) support for displaying LaTeX forumas via JavaScript (BibFormat)

 *) new and improved output formatter elements (BibFormat)

 *) new escaping modes for format elements (BibFormat)

 *) output format template editor cache and element dependency
    checker improvements (BibFormat)

 *) output formatter speed improvements in PHP-compatible mode
    (BibFormat)

 *) new demo submission configuration and approval workflow examples
    (WebSubmit)

 *) new submission full-text file stamper utility (WebSubmit)

 *) new submission icon-creation utility (WebSubmit)

 *) separated submission engine and database layer (WebSubmit)

 *) submission functions can now access user information (WebSubmit)

 *) implemented support for restricted icons (WebSubmit, WebAccess)

 *) new full-text file URL and cleaner storage facility; requires file
    names to be unique within a given record (WebSearch, WebSubmit)

 *) experimental release of the complex approval and refereeing
    workflow (WebSubmit)

 *) new end-submission functions to move files to storage space
    (WebSubmit)

 *) added support for MD5 checking of full-text files (WebSubmit)

 *) improved behaviour of the submission system with respect to the
    browser "back" button (WebSubmit)

 *) removed support for submission "cookies" (WebSubmit)

 *) flexible report number generation during submission (WebSubmit)

 *) added support for optional filtering step in the OAI harvesting
    chain (BibHarvest)

 *) new text-oriented converter functions IFDEFP, JOINMULTILINES
    (BibConvert)

 *) selective harvesting improvements, sets, non-standard responses,
    safer resumption token handling (BibHarvest)

 *) OAI archive configuration improvements: collections retrieval,
    multiple set definitions, new clean mode, timezones, and more
    (BibHarvest)

 *) OAI gateway improvements: XSLT used to produce configurable output
    (BibHarvest)

 *) added support for "strong tags" that can resist metadata replace
    mode (BibUpload)

 *) added external OAI ID tag support to the uploader (BibUpload)

 *) added support for full-text file transfer during uploading
    (BibUpload)

 *) preserving full history of all MARCXML versions of a record
    (BibEdit, BibUpload)

 *) XMLMARC to TextMarc improvements: empty indicators and more
    (BibEdit)

 *) numerous reference extraction tool improvements: year handling,
    LaTeX handling, URLs, journal titles, output methods, and more
    (BibEdit)

 *) new classification daemon (BibClassify)

 *) classification taxonomy caching resulting in speed optimization
    (BibClassify)

 *) new possibility to define more than one keyword taxonomy per
    collection (BibClassify)

 *) fixed non-standalone keyword detection, thanks to Annette Holtkamp
    (BibClassify)

 *) new embedded page generation profiler (WebStyle)

 *) new /help pages layout and webdoc formatting tool (WebStyle)

 *) new custom style template verification tool (WebStyle)

 *) added support for the XML page() output format, suitable for AJAX
    interfaces (WebStyle)

 *) introduction of navigation menus (WebStyle)

 *) general move from HTML to XHTML markup (all modules)

 *) fixed alert deletion tool vulnerability (WebAlert)

 *) do not advertise baskets/alerts much for guest users; show only
    the login link (WebSession)

 *) password reset interface improvements (WebSession)

 *) new permanent "remember login" mechanism (WebSession, WebAccess)

 *) local user passwords are now encrypted (WebSession, WebAccess)

 *) new LDAP external authentication plugin (WebAccess)

 *) new password reset mechanism using new secure mail cookies and
    temporary role membership facilities (WebAccess, WebSession)

 *) added support for Single Sign-On Shibboleth based authentication
    method (WebAccess)

 *) new firewall-like based role definition language, new demo
    examples (WebAccess)

 *) external authentication and groups improvements: nicknames,
    account switching, and more (WebSession, WebAccess)

 *) task log viewer integrated in the task monitor (BibSched)

 *) new journal creation module (WebJournal)

 *) new generic statistic gathering and display facility (WebStat)

 *) deployed new common email sending facility (MiscUtil, WebAlert,
    WebComment, WebSession, WebSubmit)

 *) dropped support for MySQL-4.0, permitting to use clean and strict
    UTF-8 storage methods; upgrade of MySQLdb to at least 1.2.1_p2
    required (MiscUtil)

 *) uncatched exceptions are now being sent by email to the
    administrator (MiscUtil, WebStyle)

 *) new general garbage collector with a possibility to run via the
    task scheduler and a possibility to clean unreferenced
    bibliographic values (MiscUtil)

 *) new generic SQL and data cacher (MiscUtil)

 *) new HTML page validator plugin (MiscUtil)

 *) new web test suite running in a real browser (MiscUtil)

 *) improved code kwalitee checker (MiscUtil)

 *) translation updates: Spanish and Catalan (thanks to Ferran Jorba),
    Japanese (Toru Tsuboyama), German (Benedikt Koeppel), Polish
    (Zbigniew Szklarz and Zbigniew Leonowicz), Greek (Theodoros
    Theodoropoulos), Russian (Yana Osborne), Swedish, Italian, French

 *) new translations: Chinese traditional and Chinese simplified
    (thanks to Kam-ming Ku)

 *) ... plus many other minor bug fixes and improvements

Download:
---------

  <http://cdsware.cern.ch/download/cds-invenio-0.99.0.tar.gz>
  <http://cdsware.cern.ch/download/cds-invenio-0.99.0.tar.gz.md5>
  <http://cdsware.cern.ch/download/cds-invenio-0.99.0.tar.gz.sig>

Installation notes:
-------------------

Please follow the INSTALL file bundled in the distribution tarball.

Upgrade notes:
--------------

If you are upgrading from CDS Invenio v0.92.1, then we advise you to
try the upgrade first on a development server.  There was a number of
important changes since the last stable release that might make the
upgrade possibly time-consuming in case you have to dump and reload
your bibliographic tables.  Please follow the following steps:

   - Launch the bibsched monitor and wait until all active bibsched
     tasks are finished.  Then put bibsched daemon into manual mode.

   - Stop all submission procedures and other write operations.  For
     example, you may want to stop Apache, edit httpd.conf to
     introduce a global site redirect to a temporary splash page
     saying that upgrade is in progress, and restart Apache.

   - Take a backup of your current MySQL database and your CDS Invenio
     installation directory (usually /opt/cds-invenio).  Remove your
     old /opt/cds-invenio/lib/python/invenio/magic and
     /opt/cds-invenio/var/www/help directories.

   - First of all, note that CDS Invenio v0.99.0 must use MySQL server
     at least 4.1 and the database must be running in UTF-8 mode.  If
     you have been running older MySQL server, or you have not created
     your database and data in default charset UTF-8 but say in
     Latin-1, then you must dump and reload all your tables.  In order
     to check which version and charset you have been using, you can
     run:

        $ echo "SELECT VERSION()" | /opt/cds-invenio/bin/dbexec
        $ echo "SHOW CREATE DATABASE cdsinvenio" | /opt/cds-invenio/bin/dbexec
        $ echo "SHOW CREATE TABLE bib10x" | /opt/cds-invenio/bin/dbexec

     If you need to convert your tables to UTF-8, then you basically
     have to dump the tables, change encoding of the file, and load
     the content back into new table definitions.  This process may
     take some time and carefulness.  See for example this blog post:
     <http://www.oreillynet.com/onlamp/blog/2006/01/turning_mysql_data_in_latin1_t.html>.

   - Second of all, you must upgrade some Python modules, namely your
     MySQLdb module as indicated in the INSTALL file.  Note that this
     will make your current site unusable, but we have stopped Apache
     already anyway.  If you have been using Stemmer, you must upgrade
     its version as indicated in the INSTALL file.

   - Untar new sources and rerun configure with old prefix argument
     (your old configure line is available at
     /opt/cds-invenio-BACKUP/etc/build/config.nice) and install CDS
     Invenio (see INSTALL file, part 1a).

   - CDS Invenio v0.99.0 uses a new INI-style configuration file
     invenio.conf instead of the old config.shtml.  Please see INSTALL
     file, part 1b on how to customize your installation.  You will
     have to create a 'invenio-local.conf' file where you have to
     merge your old config.shtml values (they are available at
     /opt/cds-invenio-BACKUP/lib.shtml/invenio/config.shtml).  Beware of
     variable name updates, e.g. 'filedirsize' now became
     'CFG_WEBSUBMIT_FILESYSTEM_BIBDOC_GROUP_LIMIT'.

   - If you have previously customized your page header and footer via
     WML, then you should now put your page customizations in a new
     template skin.  Please see the WebStyle Admin Guide for more
     information.  Also, if you have edited v0.92.1 CSS style sheet,
     you may want to merge your changes into new elements of the
     0.99.0 style sheet.

   - If you have customized your /opt/cds-invenio/etc/ files in the
     last installation (for example to edit the stopwords list or
     configure language stemmers), then you have to restore your
     changes into corresponding /opt/cds-invenio/etc files.

   - Update your database table structure:

        $ make update-v0.92.1-tables

     If you are upgrading from previous CDSware releases such as
     v0.7.1, then you may need to run more update statements of this
     kind, as indicated in previous RELEASE-NOTES files.  (You could
     also start afresh, copying your old data into a fresh v0.99.0
     installation.)

   - If you have previously customized your page templates
     (e.g. webbasket_templates.py), then check for changes the new
     version brings:

        $ make check-custom-templates

          This script will check if your customized page templates
          (see the WebStyle Admin Guide) still conform to the default
          templates.  This gives you a chance to fix your templates
          before running 'make install', by providing a list of
          incompatibilities of your custom templates with the new
          versions of the templates.

          This step is only useful if you are upgrading and if you
          have previously customized default page templates.

   - You have also to run manually several migration scripts:

        $ python ./modules/webaccess/lib/collection_restrictions_migration_kit.py

           This script will migrate restricted collections that used
           Apache user groups to a new firewall-like role definition
           language.

        $ python ./modules/websubmit/lib/fulltext_files_migration_kit.py

           This script will check and update your fulltext file
           storage system to the new style (e.g. unique document
           names).

        $ python ./modules/websession/lib/password_migration_kit.py

           This script will update your local user table in order to
           use encrypted passwords for more security.

   - CDS Invenio v0.99.0 uses new, faster word indexer.  You will have
     to reindex all your indexes by running:

        $ /opt/cds-invenio/bin/bibindex -u admin -R

   - You have to edit your httpd.conf in order to make Apache aware of
     new URLs.  Please run:

        $ /opt/cds-invenio/bin/inveniocfg --create-apache-conf

     and check differences with your current setup and edit as
     appropriate.

   - Restart Apache and check whether everything is alright with your
     system.

     Note that the detailed record pages (/record/10) have now a
     tab-style look by default.  You may want to update your formats
     to this style or else to disable this feature.  Please see the
     WebStyle Admin Guide for more information.

   - Put the bibsched daemon back into the automatic mode.  You are
     done.

Further notes and issues:
-------------------------

 *) Some modules of this release (e.g. mail submission system) are
    still experimental and not yet activated.  You may have a peek at
    what is planned, but please do not rely on them.

 *) The admin-level functionality of several modules is not fully
    developed or documented yet.

What's next:
------------

 *) Improving the known issues mentioned above.  Strengthening the
    documentation towards the v1.0.0 release.

 *) Improving the inveniocfg tool with automated upgrade capabilities.

 *) Deploying the new URL schema for all pages (admin).

- end of file -