EVT 3 alpha available for download and testing!

A.k.a. the “It’s almost Christmas!” release

After a long period of testing and bug-fixing today we release EVT 3 alpha! This is the very first version after switching from AngularJS to Angular as a development framework: the basic features are already there (namely support for diplomatic and critical editions, page-level image text linking, support for named entities), but they are still to be completed (e.g., the critical edition apparatus frame has not yet been implemented) and some basic features such as a search engine are missing as well. These features will be added during the next months, to know more about what we plan for the EVT 3 releases see R. Rosselli Del Turco et al. 2022. “There and back again: what to expect in the next EVT version”. In F. Ciracì et al. (eds.), AIUCD 2022 – Digital cultures. Intersections: philosophy, arts, media. Proceedings of the 11th national conference, Lecce, 1-3 June 2022: 212-217. http://amsacta.unibo.it/6848/.

You can download it here and leave your feedback as comments on this discussion page, thank you in advance! Note that, besides the particularly detailed README.md, there is a preliminary manual available in the doc directory.

As usual, all the source code is available on the GitHub repository.

An all new codebase using Angular

The main reason why it took so long to publish a new version of EVT is the move from the EVT 2 development framework, based on AngularJS, to the new Angular framework. Basically, we had to rewrite everything from scratch, and as you can imagine this took a loooong time. Add the COVID emergency, and you get the picture.

This has been the second code rewrite, the first one being the change from an XSLT-based architecture (EVT 1) to one based on AngularJS (EVT 2), and one we hadn’t anticipated. In fact, the development team has considered the idea of sticking to AngularJS since it served well the purposes of EVT 2. Unfortunately, on July 1 2018 AngularJS entered a 3-year Long Term Support (LTS) period, later extended to December 31, 2021: during this period the framework has not been improved and only critical functional or security flaws have been addressed; at the end of the LTS period any further support has stopped. This clearly posed a serious problem for the continued development of EVT 2.

Meanwhile, AngularJs had been replaced by a new version of the framework that is just named Angular. This new framework incorporates most of the good software design patterns and encourages good practices with regard to web application development, bringing the updated standard technologies to the table (such as the Typescript language, a well integrated testing framework, container deployment through Dockerization, etc.).

For these reasons, we decided that rewriting the EVT code using the new Angular framework would benefit the future of the project by granting the time to re-establish the software foundations on an architecture that will be able to address the flexibility required for the implementation of new features. A codebase restart also allowed us to streamline the contributions to the software, making the process more distributed and subject to the scrutiny of more people, so that a larger group of people could understand and approve the changes. More in detail, we adopted a Continuous Integration/Continuous Delivery (CI/CD) pipeline that avoids the integration of faulty code, along with the git-flow model coupled with code reviews to encourage good practices and to improve the quality of the codebase in view of the future new requirements for EVT. Finally, the new codebase management requires to trace every change to the code, thus allowing the traceability of the features.

The current architecture

The new Angular-based version is called EVT 3 and is in active development. One notable consequence of adopting the new Angular framework is reflected by the new XML/TEI parser that has been implemented in EVT 3. The parser is in charge of transforming one or more valid XML/TEI documents into a Typescript Object Model that represents the actual subset of encoded information EVT is able to compute and visualize (at the moment it mostly corresponds to a subset of the TEI schemas).

The parser itself has been designed to be independent from the actual viewer in order to achieve two important goals: (1) independence from the XML/TEI encoding, which opens the possibility of future encodings to be accepted as input data (e.g. new versions of the XML/TEI, TAGML, or others); (2) independence from the edition type and/or level (i.e. diplomatic/interpretative editions, critical editions, integrated editions etc.).

The EVT 3 architecture is also designed to be as modular as possible in order to allow the arrangement of smaller building blocks that cooperate to the construction of new views that, in turn, will manage the visualisation of different levels of edition.

Features of the alpha version

Our first goal for EVT 3 is to reach feature parity with EVT 2, but note that at the moment only a subset of the desired functionality is available, namely:

• basic support for diplomatic editions: visualization of transcription elements, editorial elements (<supplied>), special characters (TEI gaiji module), two edition levels (diplomatic and interpretative), image viewing (thanks to Openseadragon), text-image linking at page level, automatic image miniature generation and navigation

• navigation toolbar

• basic support for critical editions: inline critical apparatus only, nested apparatus entry support, critical and comment notes, pinboard area

• <teiHeader> and <msDesc> visualization

• support for named entities, including TEI special named entity lists

• <xi:include> support

As you can see, there’s a lot yet to do! But, apart from new features which will be introduced in the following releases, there are already a few new things notable even in this early release:

• themes! see above the difference between the Dream of the Rood and the Pelavicino sample editions, just choose a different theme from the main menu; this feature will be developed in the future to cater special needs (e.g. further edition customization, accessibility issues)

• a new configuration method based on three main files in the config directory, so that it is possible to share part of the configuration options among multiple editions (see the documentation in the doc directory)
• also note that the directory structure is slightly different, you now have an assets directory which will host your edition data, while the core directory shouldn’t be touched
• in diplomatic editions you can now decide to show poetic texts as verses besides the traditional document line-based visualization

• frames are now expandable and draggable, so that you re-configure the edition layout

Known bugs and issues

Since this is a first, experimental release there are probably bugs and glitches that need to be fixed, please report them together with your comments and/or enhancement requests! We have done several checks on the code, however, and it has proved to be quite stable.

Please note that it will take more work to implement the missing features and polish the overall result, which means that there will be some time to wait before a proper 1.0 version is out, but rest assured it will be worth it! So please bear with us and continue sending us feedback 🙂

Important note about testing a local installation

In the past, we always suggested that you use Firefox to test an EVT-based edition on your computer, but during 2019 things have changed: starting from version 67 Firefox developers adopted the same security-conscious policy chosen by developers of Chrome and other Web browsers, that is forbidding loading local files (= documents available on the user’s computer drive) in the browser as a result of the execution of JavaScript programs. The goal is to improve global security when browsing the Web, but the unpleasant collateral effect is that of preventing the loading of digital editions based on EVT, or similar software, from local folders. The old “launch Chrome from the command line with the –allow-file-access-from-files parameter” solution, which happened to be the simplest one, doesn’t seem to work anymore with EVT 3, fortunately there are other workarounds that can be used to test EVT editions that are located on your hard drive:

• option no. 1: download and install Firefox ESR v. 60: this version predates the new security policy adopted in FF v. 67 and, furthermore, it can be installed in parallel with any other version of Firefox;
• option no. 2: install an extension providing a local web server on Firefox or Chrome, f.i. there is this one available for Chrome:

https://chrome.google.com/webstore/detail/web-server-for-chrome/ofhbbkphhbklhfoeikjpcbhemlocgigb

• option no. 3: use http-server to set up a local server as explained here; this is particularly fast and easy if you have already node.js installed, otherwise the first step will have to be download and install node.js

This problem, however, only affects local testing, after the edition has been uploaded on a server there are no problems in accessing it with any of the major browsers.

Contact

As usual, user feedback is very much appreciated: please send all comments, suggestions, bug reports, etc. to evt.developers@gmail.com or leave a comment on this discussion page. Also, if you use EVT (either version 1 or 2 at this point, but stay tuned for EVT 3 to actually become useful) for your work we would love to know about it!

Acknowledgements

The features in this release and the current development of EVT 3 is the work of:

• Chiara Di Pietro (general development lead, tireless bug fixing and improving of implemented features, new features such as named entities support)
• Giulia Cacioli (search engine implementation, miscellaneous improvements, bug fixes)
• Simone Zenzaro (GitHub repository organization, general coordination and advice)
• Federica Spinelli (miscellaneous improvements, EVT-as-a-service development)
• Giacomo Cerretini (user annotation, semi-structured data processing)
• Sara Maenza (multiple <msDesc> and image version support)
• Andrea Caleo (VisColl support)
• Livio Bioglio (bibliography parsing and visualization)
• Elisabetta Magnanti (beta-testing, feature design, UI visualization design)
• Roberto Rosselli Del Turco (general coordination, encoding of sample documents, documentation, testing, developer nagging)

 

Announcing the second beta version of EVT 2!

After we published the first beta version of EVT 2, quite popular but still focused on the critical edition support only, development continued on many different fronts: catching up with EVT 1 with regard to diplomatic support, in particular, was high on our priority list. We are particularly happy, therefore, to release a second beta version for EVT 2 that catches up with EVT 1.3 under almost all that regards diplomatic edition support! But this is not the only new feature, here is a short list of all that goes into EVT 2 beta 2:

• [NEW] support for diplomatic and interpretative editions: all the main features available in EVT 1, such us an advanced image viewer, text-image linking, support for <choice> elements, etc. are now implemented in EVT 2 as well;

• [NEW] navigation bar: a navigation bar similar to the one available in EVT 1 has been added;
• [NEW] TOC page: a new table of contents page has been added to the general menu, to improve organization and presentation of introductory material;
• [NEW] textual search engine: complete with virtual keyboard, precise word search, highlight while you type in the search field and case sensitive search this is a great addition for diplomatic/interpretative editions;

• [NEW] VisColl support: another feature to catch up with EVT 1, the VisColl XSLT style sheet has been added to the standard EVT 2 installation so that it is now possible to prepare a description of manuscript structure and have EVT automatically create SVG diagrams which can be browsed in a dedicated frame;

• [NEW] 3DHOP integration: now you can have a view dedicated to present and browse 3D models thanks to the 3DHOP software; this is a very recent development, still in an experimental state, but you are welcome to play with it! after this release work will start again to add 3D model – text linking;

• [NEW] a Web interface for easy configuration is available for experimental purposes at this URL: http://evt.labcd.unipi.it/evt2-config/; note, however, that there have been so many changes that some of the options are not yet present in the web app;
• [NEW] EVT 2 manual available in the doc folder: this is very much a work in progress, incomplete in some parts, but may be already be useful to our users;
• [IMPROVED] named entities support: this feature has been refined so that it can work with diplomatic editions, also it is now possible to set named entities as entries in the pin area;

• lots of bug fixes and minor enhancements.

The release can be downloaded from the usual Sourceforge page or directly from the home page, all the source code is available on GitHub. You can also browse the sample edition(s) directly from the project home page.

Known bugs and issues

We’ve come a long way, but please remember that this is a beta release: EVT 2 is not by all means a finished product, and while it can be used to publish digital editions there are still some bugs left and some features missing which will have to wait until the next release. The most significant issues are as follows:

• some EVT 1 features, such as support for translations and for verse visualization, are not yet implemented in EVT 2;
• text in the <front> element is going to be shown in the Info frame, there is no support yet for <titlePage> and/or original document content;
• textual search is not working in the critical edition level, moreover for diplomatic/interpretative editions text indexes have to be created every time the edition is loaded;
• named entities lists are not supported in the <back>, only in <sourceDesc>;
• entries in named entities lists may be not shown if too many;
• full screen 3DHOP view shows a distorted 3D model.

All the issues concerning EVT 2 are available here, please do not hesitate to comment on existing ones and/or adding new ones. This tool is also built thanks to the feedback that we receive from our users.

Please note that while there will be a further release after this one, to fix more bugs and add a couple of features, the development team is currently busy porting all the code base on Angular 9! Which means that there will be some time to wait before a 1.0 version is out, but rest assured it will be worth it! So please bear with us and continue sending us feedback 🙂

Important note about testing a local installation

In the past, we always suggested that you use Firefox to test an EVT-based edition on your computer, but during 2019 things have changed: starting from version 67 Firefox developers adopted the same security-conscious policy chosen by developers of Chrome and other Web browsers, that is forbidding loading local files (= documents available on the user’s computer drive) in the browser as a result of the execution of Javascript programs. The goal is to improve global security when browsing the Web, but the unpleasant collateral effect is that of preventing the loading of digital editions based on EVT, or similar software, from local folders. Fortunately there are several workarounds that can be used to test EVT editions that are located on your hard drive:

• option no. 1: launch Chrome from the command line with the –allow-file-access-from-files parameter, after that you can press CTRL+O to open the index.html file, or you can just drag and drop it on Chrome’s window; this is probably the most simple way to do it;
• option no. 2: download and install Firefox ESR v. 60: this version predates the new security policy adopted in FF v. 67 and, furthermore, it can be installed in parallel with any other version of Firefox;
• option no. 3: install an extension providing a local web server on Firefox or Chrome, f.i. there is this one available for Chrome:

https://chrome.google.com/webstore/detail/web-server-for-chrome/ofhbbkphhbklhfoeikjpcbhemlocgigb

This problem, however, only affects local testing, after the edition has been uploaded on a server there are no problems in accessing it with any of the major browsers.

Contact

As usual, user feedback is very much appreciated: please send all comments, suggestions, bug reports, etc. to evt.developers@gmail.com. Also, if you use EVT 1/2 for your work we would love to know about it!

Acknowledgements

The features in this release have been implemented by:

• Chiara Di Pietro (general development lead, tireless bug fixing and improving of implemented features, new features such as named entities support)
• Angelo Del Grosso (Openseadragon image viewer implementation, text-image linking, miscellaneous improvements)
• Federica Spinelli (3DHOP view implementation, miscellaneous improvements, bug fixes)
• Jurgen Memaj (3DHOP preparatory work)
• Chiara Martignano (table of contents, initial support for double-end-point attached method, miscellaneous improvements)
• Giulia Cacioli (search engine implementation, miscellaneous improvements, bug fixes)
• Francesco Donzella (initial VisColl implementation)
• Simone Zenzaro (GitHub repository organization, general coordination and advice)
• Roberto Rosselli Del Turco (general coordination, encoding of sample documents, documentation, testing, developer nagging)

Many thanks to all those who contributed with feedback and advice, a special thank you to Marco Callieri for helping us with 3DHOP integration.

 

 

 

EVT 1 version 1.3 released!

The Xmas 2019 release

Dear EVT users,
after a long wait, EVT version 1.3 is finally available! Besides new features and bug fixes, we moved the repository to GitHub to make it more accessible: this is going to be the last version of the 1.* series, so enjoy the goodies, but also have a look at EVT 2 because that’s where all future development is going to have place!

You can quickly download the archive here (and the manual here), but you can also browse the sample editions directly on the project home page:

• DOTR: http://evt.labcd.unipi.it/demo/evt_v1-3/dotr/
• Codice Pelavicino: http://evt.labcd.unipi.it/demo/evt_v1-3/cp/
• Gherardi: http://evt.labcd.unipi.it/demo/evt_v1-3/gti/
• Embedded Transcription: http://evt.labcd.unipi.it/demo/evt_v1-3/et/

New and improved features

These are the main features for this release, those added since the previous release are preceded by a [NEW] label:

• [NEW] verse visualization: poetic texts written as prose in diplomatic editions are now viewable as verses in the same edition frame, numbering of verses is now present in the standard prose-like text;
• [NEW] translations: support has been added for translations of the edited text in multiple languages;

Old English text and Italian translation

• [NEW] VisColl support: EVT has integrated the VisColl XSLT style sheets so that it is now possible to prepare a description of manuscript structure and have EVT automatically create SVG diagrams in a dedicated frame;
• [NEW] chronological index: editions which are based on multiple s can now be indexed chronologically using ;
• [NEW] navigation bar: a new navigation bar has been added, which will be particularly useful for big manuscripts counting many folios;

VisColl quire structure and the new navigation bar

• [NEW] comment and critical notes: now EVT distingueshes critical notes from general, comment notes thanks to different exponents (different background color, alphabetic vs. numerical exponents); this will make it easier to create critical editions of codex unicus texts, or even of works with a very small number of witnesses;

Verse numbering and critical / comment notes

• [NEW] A Web interface for easy configuration is available at this URL: http://evt.labcd.unipi.it/evt1-config/;
• [IMPROVED] named entities support: now there’s a simple and powerful method to handle named entities, also it is now possible to define custom interesting elements which are going to be highlighted in the text;
• [IMPROVED] support for : now and its sub-elements are supported;
• many bug fixes and minor enhancements.

Sample documents

In this release the default example document is the Old English poem The Dream of the Rood, one of the texts encoded for the Digital Vercelli Book project which has been adapted to show all the new features available in this release. To better understand how the named entities work, however, you are advised to look into the Codice Pelavicino Digitale sample document (ed. by Enrica Salvatori). Finally, besides the usual TEI-based examples, we added a new document which is a play from Gherardi’s work, Le Théâtre Italien collection of plays (ed. by Barbara Sommovigo).

End of development cycle for EVT 1

As mentioned above, please note that this is the final release for EVT 1.*: after a short period to gather feedback and fix possible bugs in this version, the source code – which has been moved to a public repository on GitHub – is now fully shared with the user community to be modified and/or enhanced as other developers may see fit. From now on, all development efforts will focus on EVT 2 to reach feature parity with EVT 1 (with regard to diplomatic editions) and to improve / develop new features, starting from support for critical editions.

In any case, EVT 1.3 is a mature, featureful product which can be used “as is” for diplomatic / single witness editions, or can be further customized by anyone who has good knowledge of XSLT 2 and HTML/CSS.

Important note about testing a local installation

In the past, we always suggested that you use Firefox to test an EVT-based edition on your computer, but during 2019 things have changed: starting from version 67 Firefox developers adopted the same security-conscious policy chosen by developers of Chrome and other Web browsers, that is forbidding loading local files (= documents available on the user’s computer drive) in the browser as a result of the execution of Javascript programs. The goal is to improve global security when browsing the Web, but the unpleasant collateral effect is that of preventing the loading of digital editions based on EVT, or similar software, from local folders. Fortunately there are several workarounds that can be used to test EVT editions that are located on your hard drive:

• option no. 1: launch Chrome from the command line with the –allow-file-access-from-files parameter, after that you can press CTRL+O to open the index.html file, or you can just drag and drop it on Chrome’s window; this is probably the most simple way to do it;
• option no. 2: download and install Firefox ESR v. 60: this version predates the new security policy adopted in FF v. 67 and, furthermore, it can be installed in parallel with any other version of Firefox;
• option no. 3: install an extension providing a local web server on Firefox or Chrome, f.i. there is this one available for Chrome:

https://chrome.google.com/webstore/detail/web-server-for-chrome/ofhbbkphhbklhfoeikjpcbhemlocgigb

This problem, however, only affects local testing, after the edition has been uploaded on a server there are no problems in accessing it with any of the major browsers.

Contact

As usual, user feedback is very much appreciated: please send all comments, suggestions, bug reports, etc. to evt.developers@gmail.com. Also, if you use EVT 1/2 for your work we would love to know about it!

Acknowledgements

The features in this release have been implemented by:

• Chiara Di Pietro (general development lead, tireless bug fixing and improving of implemented features, new features such as named entities support)
• Rocco Russo (verse numbering, visualization of poetic text as verses)
• Greta Musso (VisColl support, navigation bar)
• Chiara De Martin  (chronological index)
• Federica Spinelli (support for translation, new exponents for critical/comment notes)
• Giulia Cacioli (refinement of search engine, miscellaneous improvements, bug fixes)
• Karel Pacovský (czech localization)
• Roberto Rosselli Del Turco (general coordination, encoding of sample documents, documentation, testing, developer nagging)

Many thanks to all previous contributors (in particular: Raffaele Masotti, Julia Kenny and Jacopo Pugliese) and to all those who submitted bug reports and feature requests. Alberto Campagnolo helped us understand how VisColl works and helped Greta Musso while working on the EVT implementation and on her dissertation, thank you!

 

 

EVT 2 beta 1 available for download!

The Halloween release

It’s been more than a year after the initial release of the new version, but we think that it was worth waiting for it, just look at the new features! As usual the new release will be available from the SourceForge repository, but if you’re interested in the bleeding edge code it is all available on GitHub.

This release is the result of the hard work of Chiara Martignano (critical edition features), Maurizio Ricci (bibliographic entries handling), Roberto Rosselli Del Turco (documentation, testing), Chiara Di Pietro (general development overview, repository management, bug fixing).

Quite a number of new features

These are the main features available in EVT 2 beta, those added since the original alpha release are preceded by a [NEW] label:

• Critical edition support: rich and expandable critical apparatus, variant heat map, witnesses collation and variant filtering are some of the main features developed for the critical edition.
• Bookmark: direct reference to the current view of the web application, considering view mode, current document, page and edition level, eventual collated witnesses and selected apparatus entry.
• High level of customization: the editor can customize both the user interface layout and the appearance of the graphical components.
• [NEW] Critical edition features: support for critical apparatus, apparatus fontium and analogues in a dedicated text frame; support for multiple recensio editions; support for showing text sources in a parallel frame.
• [NEW] Working area: now you can pin apparatus entries in a “pin frame” which will allow you to collect the most relevant ones for your research purposes, and to use them as bookmarks to quickly reach the desired parts of the critical text.
• [NEW] Named entities support: basic support for named entities and lists, see the example from the Codice Pelavicino Digitale.
• [NEW] Bibliography handling: bibliographic entries in a <listWit> can now be sorted by date, author or title, and they can be formatted in MLA or Chicago styles, ready to be copied and pasted in your working notes.

Available examples and configuration

This release comes with a few examples to show the new features and different use cases:

• Example 1: A short extract of Edizione Logica Avicennae, changed by CM for EVT testing purposes. It presents multiple levels of apparatuses (critical entries, sources and analogues), displayed in a separate dedicated frame.
  • Filename: avicenna.xml
  • Configuration file: config_avicenna.json

• Example 2: Pseudo edition for demonstration and testing purposes only, originally encoded by Marjorie Burghart for her TEI Critical Toolbox software, and modified in order to cover the highest number of possible use cases. It presents just the critical apparatus entry, displayed inline, within the main text.
  • Filename: pseudoEdition.xml
  • Configuration file: config_pseudoEdition.json

• Example 3: Short extract of the Codice Pelavicino Digitale edition (http://pelavicino.labcd.unipi.it/), which presents the encoding of named entities, in particular person, place and organization names.
  • Filename: pelavicino.xml
  • Configuration file: config_pelavicino.json

• Example 4: Short extract of The Tragedie of Doctor Faustus (B text) by Christopher Marlowe. Text provided by Perseus Digital Library, with funding from Tufts University. Original version available for viewing and download at http://www.perseus.tufts.edu/hopper/.
  • Filename: marlowe.xml
  • Configuration file: config_marlowe.json

The first example (Logica Avicennae) is by far the most complete and complex example with regard to critical edition support, but the pseudo-edition file is also a good way to check how how the TEI XML encoding of a critical edition works with EVT.

The third example, on the other hand, has been included to show that the porting of EVT 1 features is under way: at the moment you can see basic support for a diplomatic / interpretative edition and for named entities, but of course this is just the beginning. During next year a new image viewer will be implemented in EVT 2, so that it will be possible to add full support for diplomatic editions accompanied by digitized manuscript images.

Note that these are good starting point to start editing the config file for your edition, we advise that you keep the original example files for future reference, though.

Finally, we are also working on a web-based front-end for the configuration file, you can see and test a first version here:

http://evt.labcd.unipi.it/evt2-config/

Let us know how this tool worked for you!

Known issues

• There may be some small glitches in text formatting, such as an extra white space before punctuation marks.
• Similar errors in text formatting are present in the bibliographic entries.
• When a version text is selected by clicking on the link inside of the entry panel (i.e. “Version B”), the view mode switches to the versions view and the selected version opens next to the main text box, but the two texts don’t scroll to the selected version entry.

Future development

Work has already started on the pre-1.0 version of EVT 2: this will be the first version to include all the features currently available in EVT 1 but not in the new Angular-based version, first and foremost a new powerful image viewer and a brand new search engine! Stay tuned for further announcements next year.

Caveat

The announcement says it all: this is a beta release, i.e. while reasonably working and stable it surely isn’t bug-free, and it hasn’t been optimized yet, so you may experience slow-downs and the occasional freeze. If you find a bug please let us know! Also please remember that Chrome and Chrome-based browsers need to be launched with the –allow-file-access-from-files parameter to let them open the edition file(s) in EVT. If you want to save this hassle just install Firefox.

Another important caveat is the one about the User Interface: many new features have been added, but the UI has not yet been refined, so it may look a bit unfriendly right now. There will be a global UI redesign when we will merge all EVT 1 features in the new version, so stay tuned for a second beta announcement in 2018.

Contact

As usual, user feedback is very much appreciated: please send all comments, suggestions, bug reports, etc. to evt.developers@gmail.com. Also, if you use EVT 1/2 for your work we would love to know about it.

 

 

EVT version 1.1.1 and version 2.0 alpha released!

Hello everybody,
we are back at you with what may be called a “double feature” release: two new versions of EVT at the same time! Here are the details.

EVT 1.1.1

This version is a follow-up to v. 1.1, which was released for the DH2016 conference, and adds support for yet another specific list (<listOrg>) besides a little refining. The 1.1 and following version was supposed to be a bug fix release, but we actually managed to include new features in it: EVT 1 is going to stay with us for a long time, which means that we definitely want to support it not only with bug fixes, but also with improvements when possible. Do not expect radical changes or brand new features in this version, however.

Here are the most important changes, please read the first item carefully:

• changed the starting point of the transformation which is now based on <text> (also when using a <group> element): this is the most important difference compared to EVT 1.0, in fact note that the old method based on <div> does not work anymore, so please make sure to read the EVT 1.1.1 manual on this point;
• added support for image formats other than JPEG;
• added German language support thanks to Christian Schuster from Transylvania Digital Humanities Centre (DigiHUBB: http://centre.ubbcluj.ro/digihubb/) who actually translated the English localization file;
• added a template to rend italicized <title>s in <note>s;
• added files for custom XSLT templates (evt_builder-custom-templates.xsl) and CSS rules (evt_builder-custom-styles.css), check section 3 of the manual;
• added a distinction between critical <note>s and comment <note>s;
• renamed evt_builder-miscellaneous.xsl into evt_builder-general.xsl;
• added initial support for original content encoded into a <front>;
• added support for <listOrg>;
• many bug fixes!

Download it from the usual place and please report to us any problem that it may present. Your feedback is very much appreciated, as usual.

EVT 2.0 alpha

As the “alpha” tag suggests, this is not even a beta version of the new EVT 2 version. Actually it is very stable and does what it was created to do (support for critical editions in TEI XML format) quite well, but still has to be refined and, most of all, it lacks many important features available in EVT 1, the most critical one being support for digitized manuscript images.

This is a very important release, however, because it is the first one based on the Angular.js framework and on the MVC design pattern, chosen because it allows to separate the logics of data presentation from the core of their processing. We needed more flexibility and modularity in order to make the development of new features easier and faster, and this has led to a completely new infrastructure: the XSLT stylesheets have been abandoned in favor of a set of JavaScript parsers specifically written to retrieve edition content directly from the XML file. Edition data is then saved in a JSON structure, organized in such a way that it can be easily and rapidly accessed when needed. The final styling of documents is entrusted to CSS style­sheets and is easily customizable.

Note that the current Javascript parser can only process TEI XML documents, but it is possible to add new parsers for different formats so that they are converted to the intermediate JSON structure, e.g. it is possible to add a custom XML (non­-TEI) parser, a LaTeX parser, etc.

These are the main features available in EVT 2.0 alpha:

• Critical Edition support: rich and expandable critical apparatus, variant heat map, witnesses collation and variant filtering are some of the main features developed for the critical edition.
• Bookmark: direct reference to the current view of the web application, considering view mode, current document, page and edition level, eventual collated witnesses and selected apparatus entry.
• High level of customization: the editor can customize both the user interface layout and the appearance of the graphical components.

You can go straight to the online test site, but if you want to experiment with it you only have to download the archive from SourceForge and unpack it on your hard drive, after that you click on the index.html file and you are ready to go. If you want to try one of your documents you can do this by simply copying your file in the data folder and then editing the config.json file in the config folder: make sure that the dataUrl variable points to your file, e.g.

"dataUrl"          : "data/my_critical_edition.xml",

This version is the focus of all new development, so please let us know what you think of it writing at the usual contact email. Your feedback is even more appreciated for EVT 2 because this is the first public version and we definitely need to know if the path we have chosen is appropriate.

Many thanks to Marjorie Burghart for the TEI Critical Apparatus Toolbox which allowed us to perform a preliminary test of the TEI Parallel Segmentation method. A modified version of its sample file is the default pseudo-edition text showed in EVT 2.0 alpha.

 

 

Second beta of the Digital Vercelli Book available!

As hinted in my previous post, a second beta of the Digital Vercelli Book published using EVT 1.0 is now available here: http://vbd.humnet.unipi.it/beta2/.

Full announcement available here:

 

EVT version 1.0 has been released!

After a couple of intermediate releases, EVT version 1.0 is finally available! There are many new features in this version:

• a full text search engine, with result highlighting in the text frame;
• support for named entities: name highlighting in the edition text, browsable full lists with links to single documents;
• support for <listPlace> and <listPerson> elements;
• support for diplomatic documents (regesto) and for general information about specific texts;
• support for bibliography, manuscript and project information;
• User Interface localization;
• lots of bugs fixed!

Go download it and check for yourself on the SourceForge site. For more information about how to use and/or customize EVT please refer to the EVT Manual included in the archive you downloaded, just look in the “doc” folder.

Please send all feedback to evt.developers@gmail.com[1], we would love to hear from you: bug reports, suggestions, even your general impressions about how EVT has worked for you are more than welcome. If you want to check an actual edition made with EVT, possibly showing all the latest features, the excellent Codice Pelavicino Digitale [2] is just what you want to browse.

Named entities highlighting in the Codice Pelavicino Digitale

Also watch this space for an announcement about the Digital Vercelli Book 😉

EVT version 1.0 is the result of months of hard work by our development team:

• Chiara Di Pietro
• Raffaele Masotti
• Julia Kenny
• Ilaria Tiezzi
• Chiara Alzetta
• Jacopo Pugliese

Thank you all and congratulations for this achievement!

RRDT

[1] You can also drop me a line at roberto.rossellidelturco@gmail.com. If for any particular reason you would prefer to send feedback anonymously, you can do it using this form; note, however, that it may take a little longer before one member of the development team checks it.

[2] The default EVT example in the 1.0 archive is in fact a single document from the Codice Pelavicino manuscript.

 

 

SDE design: The agony and the ecstasy I

During all phases of EVT development the team members have had quite a number of lively discussions about the many topics and problems that such a (relatively) complex project entails: which programming and markup languages to use, basic features to implement in the beta version, the license to adopt when releasing EVT as open source software, the roadmap leading to its first release and subsequent ones, and more. Even considering roadmap-related issues, strictly linked to the financial and human resources available (hint: never enough of both!), the most heated debates revolve about one fundamental issue: which User Interface design for Scholarly Digital Editions should be implemented in EVT. This is easily explained if you think that we share basic ideas about most, if not all, of the technical aspects of software development, and that in any case I can only listen and then give green light on most of those issues, since Chiara, Julia and Raffaele have a competence in strictly technical aspects far exceeding mine; furthermore, roadmap-related issues usually fall in a sort of “external agents” category, be they impending university exams, a deadline for external projects or papers to be delivered, or quite simply a lack of adequate funding for all the nice things we have in mind.

UI (and UX: User Experience) design, on the other hand, is both an aspect of development we all believe to be fundamental for the success of EVT, and an area where every team member has her/his own specific sensitivity/orientation, therefore each of us has her/his own vision with regard how to make EVT “right“. A UI/UX done “right” makes a difference between a research tool that can prove useful to get stuff done, and a way to enjoy exploring and reading a digital edition besides that. Everyone in the team aims at this second goal besides the first one, and everyone has his/her own ideas about how to reach it. UI/UX development for EVT, therefore, is more a bumpy country road, where sometimes you get to regret how slowly you have to walk, than a speedy motorway leading you to the final, commonly shared destination. Also, there are lots of detours, dead-end streets and, of course, suffering and whining because of all the bumps 😉

“Getting the UI/UX right” for a software such as EVT is of primary importance for a number of reasons:

• first of all, what is visible and actionable on the screen is “the edition” for our perspective users: at a markup level the edition is already there, but of course it is “invisible” for the intended public since without a tool such as EVT it wouldn’t be possible to access it, browse it, use it for research purposes or to simply enjoy viewing the manuscript images, reading the texts, and more; “getting the UI/UX right” means “getting the edition right” for all EVT users, and thence it is as important as “getting the edition right” at the markup and editorial level; note that, conversely, the marked up text is invisible to users once the edition has been published with EVT;
• as noted by several researchers (see for instance Porter 2013 and my own 2011 article), a digital edition is still at disadvantage when compared with traditional printed editions because 1. digital editions are in small or big ways all different 2. you have to learn how to use them, both because of 1 and because they often include tools, such as text search/analysis tools, that are simply not available for printed editions (and, again, are all different in some way while sharing some common UI features); “getting the UI/UX right” also means lowering the learning curve as much as possible, with the ideal goal of making the digital edition as intuitive to use as a printed one;
• last but not least, on the strictly technical level some aspects of the UI/UX are firmly connected with the internal workings of the software: a convoluted navigation system, for instance, would probably cause unnecessary stress on the software layer devoted to that specific purpose; vice versa, an elegant internal architecture would make it easier to devise an effective navigation method.

The first two points should be clear to everybody involved in the critical digital editing field, both creators and users of electronic editions. Indeed, we have received a good feedback about EVT by users of the Digital Vercelli Book beta, and generally appreciative remarks for its simple and elegant UI. Things are going to get more complicated soon (see below), but it never fails to surprise me how people may not be able to describe exactly what they what, but invariably recognize it when they’re presented with it. This is the general feeling with EVT and the VBD beta, even considering all its limitations, and it was a good one.

There’s another aspect to feedback, however, and it’s a darker, more troubling one: often the same people who appreciate a well done digital edition, and who are interested in using such resources on the Web, are not interested in taking part, if not in the actual design of such resources, in theoretical discourse about how to improve general usability and effectiveness of Scholarly Digital Editions. There is a general feeling that somehow, somewhere, somebody will come up with exactly the kind of resources that they need; not only that, but also the deeply ingrained conviction that an editor should do just that, prepare critical editions of his/her texts, and avoid at all costs getting involved with how these texts may be displayed and used in a digital edition: after all, this is why we have IT departments and technical personnel, right? not real “textual criticism” in action, is it? I will not surprise anyone by writing that I couldn’t disagree more with this line of thinking: it really is hard for me to imagine how an ICT professional, however excellent in his area of expertise, could design (not implement!) the perfect digital edition tool. Think of the “different layers …” of a critical edition, or even just the complexity of a full critical apparatus: who, if not a textual criticism scholar, is going to “get it right” when it comes to how and where the information is organized and showed to the user? Explaining to a programmer what we have done so far with printed editions, the rationale for critical apparatus etc. is not enough, since it’s the reason why so many first-generation digital editions resemble printed editions so closely, with a fixed layout similar to how the space is used on a printed page for the edition text, the critical apparatus, commentary notes etc.

For real innovation to happen in the SDE area with regard to their UI/UX component we all need to get our hands “dirty” with User Interface design: which doesn’t mean that anyone researching or studying textual criticism has to go out and buy the latest publications in HCI studies, of course, just sending detailed feedback to those creating digital editions and online resources would be a great start and a very appreciated help.

Those actually working on UI design for digital editions, on the other hand, need not only to listen to their own current and prospective users: they also need to put in place an effective way to share, discuss, accept and test UI design proposals. This is not a simple task in any research team, a fact of life that we discovered early during EVT development. The next post will be devoted to the work flow we are currently using and improving (or trying to) to design the next EVT version.

References

Porter, Dorothy C., ‘Medievalists and the Scholarly Digital Edition’, Scholarly Editing: The Annual of the Association for Documentary Editing, Volume 34 (2013) <http://www.scholarlyediting.org/2013/essays/essay.porter.html > [accessed 20 April 2015].

Rosselli Del Turco, Roberto, ‘After the editing is done: designing a Graphic User Interface for Digital Editions’, Digital Medievalist Journal, 7 (2011). <http://www.digitalmedievalist.org/journal/7/rosselliDelTurco/ > [accessed 20 April 2015].

 

TEI Embedded Transcription support in EVT

Since it was originally born as part of the Digital Vercelli Book project (http://vbd.humnet.unipi.it/), EVT was developed to deal with the XML encoding of texts which had been prepared for that project, namely making use of the XML TEI P5 parallel transcription method (http://www.tei-c.org/release/doc/tei-p5-doc/en/html/PH.html#PH-bov). When using this method, information about the scan and possibly the coordinates of sensible areas are separated from the transcription and aligned with it thanks to linking attributes.

The Vercelli Book Digital beta version using EVT

But, as it is possible to read in the TEI Guidelines, the scholar can choose to emphasize the importance of the physical surface and to encode words and other written traces as subcomponents of the XML elements representing the physical surface carrying them, rather than independently of them. This kind of encoding scheme is known as embedded transcription (www.tei-c.org/release/doc/tei-p5-doc/en/html/PH.html#PHZLAB), and thanks to support from EADH (see EADH Small Grant: Call for Proposals at http://www.eadh.org/support/eadh-small-grants-call-proposals) this feature was added to the EVT software. The development took place in the period between May and July 2014.

Main changes to the original software

The main changes we implemented are mainly related to the identification and split of the text into different folios and the creation of the structure for the image-text linking tool.

Since the Vercelli Book transcription was completely encoded according to the parallel transcription method, we used different texts in order to have proper examples of embedded transcription; in particular, we used the TEI examples available in the Guidelines (http://www.tei-c.org/release/doc/tei-p5-doc/en/html/PH.html#PHZLAB) and the encoded text of the Slovenian «Tri Pridige O Jeziku (three sermons on language)» (http://nl.ijs.si/e-zrc/slomsek/index-en.html).

The Tri Pridige O Jeziku edition, an ET transcription, using EVT

First of all, we added an automatic detection of the encoded scheme used in the text that is being transformed (Parallel Transcription or Embedded Transcription): this identification is based on the absence/presence of the <sourceDoc> element, which is only used in ET.

If the system finds at least one <sourceDoc> element, the text will be treated as being encoded in ET: thus, each <sourceDoc> will be handled as a different document and each <surface> element, both when child of <sourceDoc> and when child of a <surfaceGrp> element, will be used to generate a single textual fragment.

We decided to consider the <surfaceGrp> element just as a mere generic division inside the code that does not produce any particular output in the interface. Moreover, even if the possible nestings of <surfaceGrp> are infinite, at the present moment the software is only able to support two levels.

The most important element after <sourceDoc> and <surface> is the <zone> element. This will be used to create the elements required for the activation of the image-text linking tool and of the hotspot tool.

A <zone> can be an empty node linked to one or more textual nodes, making use of the <line> element, or it can contain the text directly, without any further sub-elements. We considered both cases, therefore the XSLT transformation for the image-text linking tool will be activated with:

• a <zone> element that contains some text and has the spatial coordinates attributes @ulx, @uly, @lrx and @lry. In this case, each sensitive area of the image identified by the previous coordinates will highlight all the text that was nested in the <zone>, even if it is distributed on more lines.

• an empty <zone> element that has the spatial coordinates attributes and a reference to the particular <line> element it is linked to. Similarly to the previous case, each sensitive area of the image identified by the coordinates will highlight the text inside the element linked to the particular <zone>.

When the <zone> is missing the spatial coordinates attributes, the text-linking tool will not work, but the corresponding text (both if it is inside or outside the <zone> itself) will be rendered in the interface and nested in a particular HTML container (<div class=” *edition_level*–Zone “>), in such a way that the user can visually distinguish the separation between the different <zone> elements; the specific class (one for each edition level configured) allows to easily customize the visualization of the <zone> on the browser.

Instead, if the <zone> element is an empty node and the reference between it and the textual node is missing or broken, the text will properly appear on the page, but the image-text linking tool will not work for it, even if the <zone> had the spatial coordinates attributes.

As said before, in some cases the <zone> element will be used to generate an HotSpot, that is a sensible area on the image directly linked to a HTML pop-up window. We have decided to consider as a hotspot:

• every <zone> that has the spatial coordinates attributes and is nested inside another <zone>; in this case the textual box will contain the text of the innermost zone;

• every <zone> (even if a direct child of <surface>) that contains a <graphic> element with a @url attribute; in this case the textual box will contain the image referenced by the <graphic> element and the text inside the <zone> itself (if present).

All hotspots that were handled by means of the @rendition attribute in texts encoded in PT, will likewise work fine with texts encoded in ET.

Future developments

The remarkable variety of possible encodings available when using the Embedded Transcription method has made the task of supporting it more complicated than we expected. As it is clear from the section above, at least in this phase of development our support is somewhat “prescriptive”, in the sense that not every possible encoding is supported. This means that text encoded according to “reasonable” principles will very likely work, while in other cases it may or may not work and thence require some modifications to the encoded text. Since this is the first version of EVT supporting the Embedded Transcription method, we expect further improvements on the basis of users’ feedback: as always, feel free to contact us with your remarks, suggestions and feature requests. We have contacted the authors of the «Tri Pridige O Jeziku (three sermons on language)» edition mentioned above, and will experiment together with them with the goal of fine tuning ET support in EVT.

Contact

EVT Project editionvisualizationtechnology@gmail.com
Roberto Rosselli Del Turco roberto.rossellidelturco@gmail.com

List of participants

Chiara Di Pietro (dipi.chiara@gmail.com)
Julia Kenny (julia.kenny90@gmail.com)
Raffaele Masotti (raffaele.masotti@gmail.com)

References

Digital Vercelli Book project: http://vbd.humnet.unipi.it/.
EADH Supported activities and reports: http://www.eadh.org/support/supported-activities-and-reports [full report submitted to EADH].
Edition Visualization Technology: http://sourceforge.net/projects/evt-project/.
Digital Vercelli Book beta version using EVT: http://vbd.humnet.unipi.it/beta.
TEI P5 Parallel Transcription: http://www.tei-c.org/release/doc/tei-p5-doc/en/html/PH.html#PH-bov.
TEI P5 Embedded Transcription: http://www.tei-c.org/release/doc/tei-p5-doc/en/html/
PH.html#PHZLAB.
Tri Pridige O Jeziku (three sermons on language), http://nl.ijs.si/e-zrc/slomsek/index-en.html.

Post based on the final report by Chiara Di Pietro and Julia Kenny, revised and modified by R. Rosselli Del Turco.

 

EVT development: an update (and quite a bit of history)

Quite a lot of time has passed since the last entry on this blog, and many things have happened with regard to EVT development. It seems to me a good idea to propose a quick recap of the latest events, starting with a bit of history first, though.

There have been several incarnations of EVT, right from the times when the idea of a Digital Vercelli Book was just taking shape and well before even an experimental encoding of the transcription had been attempted. As you can see from the images below, at the time (about 2002-03) I was thinking of a very simple viewer to show images and the corresponding transcription/commentary texts, with a few features to handle the former (magnifying lens, hot spots, graphic filters).

The reference model was that of the Electronic Beowulf edition by Kevin S. Kiern, which looked like an incredibly powerful software to me, something too complex to be effectively replicated with the available resources (at that time: no resources to speak of, actually).

After the transcription efforts were started for good, using analogue photos converted to digital images, I kept looking around and testing all digital editions I could land my hands on. I was quite impressed by the viewer created by B. Muir for the Junius Manuscript, the Electronic Exeter Book and other image-based editions: in spite of some problematic UI choices (see After the editing is done: Designing a Graphic User Interface for digital editions), it was a very effective tool, suitable for all the research activities (image analysis, text search, etc.) that I envisioned for the DVB. It came with a few limitations, though:

• very hardware and software specific: it required a specific version of a specific browser, which of course was the one available for the most widely used operating system (Internet Explorer 5.5 for Windows), but not for Linux or MacOS;

• everything was HTML-based: while a static conversion from TEI to HTML has never been a troubling prospect for me, giving up on all the semantic information slowly and carefully included in the encoded transcription was quite a big deal; note, however, that now HTML is much “smarter” and semantic-friendly (thanks to HTML5 and microdata);

• the viewer was to be licensed, being proprietary software, and while I’m sure I could have bargained a good deal about a VBD-specific customization, I’m a big supporter of the open source concept: doing something good and having its use and circulation restricted didn’t really sound right to me; in any case, have I mentioned I had few resources at the time?

Another interesting software that I had a chance to see in action at about the same time (at Kalamazoo 2005 if I remember correctly) was the Elwood viewer: it sported a very similar mix of good (fast, effective, powerful, multi-platform) and less good (proprietary, some UI issues, still in development) characteristics, so I had to rule it out, too. The good news is that development is finished and that the author is going to open source it after a final code clean up, hopefully sooner rather than later: it’s an amazing piece of software, and my EVT development leader can’t wait to have a look at the code 😉

One project that caught my attention, and with which I collaborated for a while, was the EPPT software whose development was directed by Kevin S. Kiernan: it offered very comprehensive features for creating an image-based edition, and the plan was to extend its capabilities to the edition visualization part. Unfortunately, it never received the resources necessary fulfil such an ambitious plan, and it didn’t make all the necessary progress with regard to the presentation part. At the moment the web site doesn’t seem to be reachable any more: the lead developer informed me that this is due to a new server going to replace the old one, which crashed and was retired from service. The really good news is that this software has already been open sourced and it’s available on the SourceForge platform. This is excellent because its production component is mature and very sophisticated; among other features, it allowed to overcome the infamous multiple hierarchy problem which is XML’s greatest limitation. An updated version that could save in TEI XML format would be a great asset to the digital philology community.

What about the TEI community? There have been several projects in the past, some of which have unfortunately been abandoned (TEIViewer, TeiPublisher), while others were, and still are, unsuitable for diplomatic and image-based editions (TEI Boilerplate). Furthermore, almost always I had issues with the UI, or with some other critical aspect, such as supported platforms and OSes etc., of the software. What can I say, I am that fastidious …

So, in spite of my aversion to the NIH syndrome, possibly the worst problem plaguing the open source/free software community, I started looking into a light weight, web-based solution for a digital edition of the Vercelli Book manuscript. Starting from 2008 onwards I was asked to teach text encoding at the University of Pisa, in a Digital Humanities degree course, and I came in contact with a good number of smart and talented students. It was natural, then, to propose EVT as a PBL (Project-Based Learning) subject, with the realistic purpose of having a good experimentation playground and the more distant goal of actually producing something usable. In fact, the first EVT version was already useful for an edition, so both expectations were met.

But the student who created this first version, Francesca Fiorentini, wasn’t interested in continuing its development, so I assigned the task to another student, Francesca Capochiani. EVT v. 2 was born as a completely new project, written from scratch, and looked particularly feature-rich:

Well, perhaps even too much feature-rich: Francesca worked hard and developed her version in a very autonomous manner, so basically I was showed the end result of her efforts. Just while I was grumbling about the need of a full team of developers, one that would discuss and make decisions in a cooperative way, Francesca Capochiani decided to pursue other interests and dropped all EVT-related development. Which reminded me of another advantage of a development team: if you lose a single individual, the common knowledge acquired is shared and lives on, development can continue.

I was thinking about this kind of organizational problems when I met Raffaele Masotti, who worked on EVT first for a stage and then for his BA degree. Meeting Raffaele was crucial for the destiny of EVT in more than one way:

• first of all, Raffaele brought in a careful, meditated approach: see what is working in the current code base, decide which features to put aside (at least for the moment), examine the several problems which still had to be solved; for me, who feared another “let’s rewrite it from scratch!” jump in the dark, this was quite a relief;

• one of the problems that still had to be solved was quite a fundamental one: what is the best way to load your data in a web-based viewer? after quite some study and research, inspired by the recent TEI Boilerplate and another project adopting an XSLT-based strategy, Raffaele turned that question upside down in what is EVT’s approach: you don’t do that, you build the viewer around the data; that’s how EVT Builder was born;

• so we ended up rewriting EVT from scratch, after all, but it was for a very good reason, a complete re-design of its basic architecture that proved to be simple and effective;

• last but not least, Raffaele proved to be very capable in handling EVT development under several aspects: when a dev team was eventually created by accepting more students for EVT-related stages and dissertations, Raffaele took care of introducing his colleagues to the software’s inner workings, of assigning tasks, of merging inputs in the code tree and more.

This last point is particularly important because the dev team has grown quite a lot: first we had Julia Kenny implementing two fundamental features, the magnifying lens and the image-text linking; she also did an excellent job at modifying the XSLT build chain, in such a way that she can be considered a co-author of the software. After Julia we added Jacopo Pugliese and Chiara Di Pietro to the team, dealing with the search functionality and critical edition support, respectively. The four of them can be considered the current EVT core team, to which you could add Giancarlo Buomprisco, currently working on the Digital Lightbox functionality for the DigiPal project; hopefully we’ll see a “light” version of his work in EVT at some point during 2014.

The newly born EVT Builder was presented at the Easy Tools for Difficult Texts workshop (The Hague 18-19 April 2013). The presentation was well received, so I encouraged my valiant students to work on a poster for the TEI MM 2013 (Rome 2-5 October 2013). The TEI Conference was a great occasion to meet people and show our work, and again we got a fair bit of attention, but it was thanks to the support received by the Biblioteca Capitolare that we could push for a an acceleration in EVT development, which led to the beta release of the Digital Vercelli Book shortly before Christmas 2013. The weeks before it were quite hectic: we only met a few times in person, but all the team was constantly in contact through an assorted set of chatting tools, mails, shared docs and even, when necessary, old fashioned phone calls! The development repository on GitLab was hit with scores of commits every day, until the software was ready for its debut. The screenshot below shows the final version of the beta, uploaded on SourceForge soon after the release.

Thanks to the hard work of all the team EVT development continues, for the moment being aimed at fixing bugs and polishing the Digital Vercelli Book beta, after which more important features will be added. But that’s the subject for future posts 🙂

RRDT