: Digital

VADU: Visual Audio Display Unit

My last blog entry was back in April 2009, so this is a hesitant return.

Background

In July 2011 the refurbished Contemporary & Modern Art galleries were re-opened at the National Museum Cardiff, the following VADU would be included in the initial exhibition. The specification was mainly to showcase video shorts: recorded, interviewed and edited by a few of my colleagues (Art and New Media), the videos would have subtitles and also there would be a visitor comments page.

The iPad 2 was about to be launched when I started work on the VADU. Magic was in the air, queues were forming.

Note: it has been used for two exhibitions, so far:

1. Contemporary & Modern Art (July 2011 – January 2012)
2. The Queen: Art and Image (January 2012 – April 2012)

Project Ethos

Although there is an 'App Store', it seemed like overkill to write and release an application simply for four machines in an art gallery - I used 'Kiosk Pro' instead, an application which basically removed all the usual iPad functionality and locked it down to a fullscreen Safari browser.

In terms of creating the backend mechanisms for providing/collecting content, one shouldn't reinvent the wheel. Therefore, I used the trusty Museum‘s web CMS (Amgueddfa CMS) to control things - the Amgueddfa CMS controls the museum websites and intranet for all seven nation museums. Built in-house over several years, it has evolved with the requirements of the organisation. It's built with open source PHP, MySQL and Javascript. Succinct and effective, even if we say so.

In regards to the actual design aspect of the VADU interface: the only two constraints were museum brand guidelines and a particular colour had to be used to unify it with the surrounding new gallery signage.

Around May 2011, Braun's designer Dieter Rams design ideas entered my world (I can't repeat his mantra here, but only because it is copyrighted). Anyhow, I tried to create an interface that was simple, intuitive, consistent, and didn't distract the user from the actual content:

• Only two colours
• No drop-shadows or gradients
• 10px borders
• I did allow myself one curvy corner (bottom-right), one guilty pleasure
• User navigation: horizontal gestures only
• Page transitions: vertical only

Building Blocks

If you haven't already looked at the photograph 1 and 1b, I would take a look now; so we have a shared reference images in mind.

The top section displays the title of the exhibition, followed by page buttons (Art, Comments and What's On). All exciting stuff - the tip of an arrow indicates which page the user is on (hopefully in a subtle fashion).

The language button starts the most convoluted process on the VADU, in terms of animating a page change. I didn't want the change to be instant, instead I gave it a more graceful flow. The actual result of changing the language only swaps the domain name from English to Welsh or visa-versa (museumwales.ac.uk, amgueddfacymru.ac.uk), but the time it takes to do that is over four seconds. It should become clearer later, if you watch the video below.

I used the Javascript Mootools framework, it has served us well - lightweight and fast. I chained the transitions and effects to create the smooth transitional actions throughout the VADU. It was also necessary to chain things because I would describe the Safari browser on the iPad as OK, in regard to the power of Javascript, rather than excellent if used on a desktop (there is reduced access to the normal power you find in apps wrapped up in objective C; no WebGL access).

The middle section displays the video in focus at the time, large screen print from the video is shown in the background - title, summary and extraneous information such as video length are shown on the information panel. In large font the word: 'Play', indicating the user can start a video. Left and right arrows also allow the user to shuffle through the videos. The information panel can be moved from the right or left of the background image - something that resulted from the fact artists don't like their work flipped [in a digital sense] i.e. if the focus of the background is to the left, the information panel can be positioned to the right by indicating such in the CMS metadata entry.

The bottom section can be dragged with a finger left and right, selecting any one of the sixteen videos. It was quite important to have draggable areas, because it is simply expected by iPad users (thus, making it intuitive). The same draggable feature is used for the what's on page (photograph 6).

If the user selects a video, the screen removes all navigation features so they are only left with the title of the video, a video time indicator (so the user knows the video is only short), the video itself in the center of the screen, the subtitles at the base of the screen and a 'back' button. The user can pause and un-pause the video by tapping of the video in the center (see photograph 3). The video's themselves use the same colour as the VADU, so it all fits together neatly.

Finally the comments page is simple too: optional name input, text input and the last three comments are displayed on the right-hand side (hopefully encouraging the user to write something). The comments are fed into the usual website comments system, approved (or not) by a staff member (photograph 5) – there has been over 2000 comments left of the gallery VADU since July 2011, which is quite a lot considering no one was forcing these people to write something.

Subtitles

The subtitles are again displayed using javascript, they are not integrated into the video. I built a subtitles tool within the Amgueddfa CMS using Javascript and Flash, which creates a XML file that could be dropped into the caption of the video. The idea being that the majority of videos need to be available to two languages and if the videos were published on the main website at anytime, a transcript could be displayed too. Hopefully the task of transcribing the video in the first place is easier because of it. There you go, a little of the behind the scenes work (photograph 7).

Past Performance

The gallery VADUs have been very reliable; once every few months one of them may freeze, but considering they are always on (one weakness of iOS software is you can‘t boot-up into a single application), that's not too bad. I darken the screen after the galleries close – simply using a whole–screen black div.

We had a brief problem when changing the local network settings in January, so I added a check before the VADU changes language to see if there is a network available (an AJAX query: onSuccess or onFailure).

Future Development

If it continues to be used, I would like to develop a local version of the VADU, providing a fallback if the network goes down, or maybe a hybrid version (storing the videos on the VADU). This would mean a update of the iOS (from 4.3 to 5.1+), but I‘m sure there would be some associated browser performance improvements.

Other major changes shouldn't be required, as the video shorts are meant to be the star of the show.

Final Word

Obviously it helps to have a pleasant environment to place the VADUs (photograph 9).

Video Demo

I've included a short demo video for posterity:

Graham Davies, Online Curator, Amgeddfa Cymru – National Museum Wales

When a member of the Art department approached me to ask if I could feature two views of the same painting online – one version covered in dirt and yellowed varnish (as the painting was when it came into the Museum), and the other version showing hidden detail and crisp colours (after being cleaned by Museum conservators) – I realised it would make a perfect interactive if you could use your mouse to virtually 'clean' the dirty canvas to reveal the clean version underneath.

Guardi's view of the Grand Canal, Venice

The painting in question is Francsesco Guardi's View of the Palazzo Loredan dell'Ambasciatore on the Grand Canal, Venice, painted around 1775-85.

Acquired by Amgueddfa Cymru in 2011, this painting is an important acquisition as Guardi's Venetian views are regarded as highly significant in the history of landscape painting.

You clean the painting

To make the most out of this dramatic before-and-after view, I needed to work out a way of 'virtually' cleaning the painting online by dragging a mouse over the dirty image to reveal the original details and colours previously hidden underneath the dirt and old varnish.

Reinvent the wheel?

I wanted something that allowed the mouse to act as an eraser, allowing one image to be rubbed out to reveal a secondary image underneath. A hunt around the internet brought up the required functionality already created by by Jonathan Nicol (www.f6design.com/journal).

The next step was to acquire high-resolution copies of both dirty (before) and cleaned (after) digital images of the artwork from the Photography department.

Precisely aligning two slightly differently angled photographs of the same picture

When I opened these digital images in Photoshop it became apparent that variations in the perspective, and distance of the photographic captures, resulted in two images that did not precisely match up once overlaid on top of one another.

After an hour of miniscule adjustments using the image warp feature on Photoshop, using the images as separate layers within Photoshop (one set at 50% opacity), I eventually achieved a precise overlaid match.

I abandoned trying to do this at 100% view as the image was so large and the time lag in processing too great to view the results (even for my G5 at 2.44Gz and 8GB RAM). I had to settle for a 25% view that filled my Apple 32" screen)

Once I had a satisfactory matched up and aligned the 'dirty' layer on top of the 'clean' layer, I could create the two corresponding TIFF images to incorporate into the Flash file as a basis for the interactive.

After a bit of tweaking, fiddling, and constant testing, I managed to create a simple interactive, allowing you to use your mouse to erase the dirt and grime, revealing the clean painting underneath.

Exploring the detail

I then decided to repeat this process to create several versions, all using crops of the high-resolution images to show close up details of the painting.

Areas of particular interest I chose to separate out were people rowing a gondola, the architectural detail of the buildings, and the detail of the sky and clouds where much original detail had been almost totally obscured by years of grime, dirt and previous 'touch-ups' to the painting. The clean version revealed original intricate details and brushwork.

Future applications for Museum archives and collections

I am hoping this functionality can be utilised for other online images of the collections in the future. Ideas I have at the moment are to reveal hidden under-drawings only visible under x-ray light — as in the example of Richard Wilson's Dolbadarn Castle (NMW A 72), which has been painted over a portrait of a woman, and Landscape with Banditti around a Tent (NMW A 69) which he painted over a Venetian-style reclining nude.

Additional ideas include viewing a landscape or post-industrial townscape that can be erased to reveal a historical image underneath...

Ok, so we had the iPad moment. What’s changed? Lots. The iPad itself was, in truth, disappointing for publishers. Beautiful, sure, but not very helpful. It wasn’t multifunctional and it wasn’t backward compatable with much stuff either (I can’t be the only person still using OS 10.4?) But, like Apple’s previous offers, it was a gamechanger. It established the tablet as a device, despite many people, myself included, wondering if anyone really wanted Job's 'third device'. Apple then let other manufacturers come up with their own versions, the best of which is probably Samsung’s Galaxy, and quietly went home to improve their own model. Having established the tablet, and just in time to catch the secondary wave of adopters, out comes iPad 2. With improved functionality and more features (camera – two, actually), it still passes itself off as the most desirable tablet, even if it’s not necessarily the best. With iPad 2 and the iPhone, Apple has now firmly entered the mainstream consumer market. In losing the geek factor, what has it gained? Well, turnover, and profit, obviously. While Apple’s top-quality combined hardware/software model of Macs retains its market-leading position in the creative industries, the iPods, Ipads and iPhones are now thoroughly high-street, even with their top-end price tags.

However, part of this trajectory has been the strategic downplaying of the iPad’s e-reader function, which is what publishers were most  excited about. Instead, the iPad focuses on portable, sleek, seamless acces to the web and email – truly, a big iPhone, but also ready and waiting for Web 3.0.

In terms of e-readers the iPad moment just didn’t happen. This has left Amazon’s Kindle as market leader, even though it only reads Amazon’s own e-book file format (although there are rumours Amazon will soon be allowing US publishers to submit e-books in the industry-standard e-Pub format). Amazon introduced the Kindle in 2007, by 2010 in the US Amazon were selling more Kindle books than hardbacks; today Amazon sells more Kindle books than hardback and paperback put together. At the moment it’s selling 105 Kindle books for every 100 print books, and three times more Kindle books than this time last year. In the UK, where the Kindle store has only been open a year or so,  Amazon are selling twice as many Kindle books as hardbacks.

What can we learn from this? Remember, the Amazon figures only apply to their own sales, of Kindle books, which can only currently be read on a Kindle device. What’s happening across the rest of the bookselling industry? The true picture for the UK is that sales of e-books are currently 2.5% of all book purchases; interestingly, they peaked at 3% over Christmas (did you get an e-book in your stocking?!) Adult fiction is still the most popular category, at 5.4% of all purchases; men and women are buying e-books equally, and the age group 55-64 makes up over a quarter of e-book buyers.

This 2.5% seems like a tiny figure for us all to be worrying so much about, especially as the value of the sales is low – about 1.6%. I still can't wait to have a go though.

Revealing the historic sketches of Francis Place for the very first time…

After Museum conservators in the Art department had completed their conservation work on the Francis Place sketchbooks – containing some of the earliest on-the-spot- sketches of Wales in the Museums collections – I was given the task of figuring out the best way in which to present these sketches online.

Secret sketches hidden for 200 years

Places' sketchbooks had been taken apart 200 years ago and their pages stuck on a woven paper backing. Recent conservation work has since revealed further sketches on the reverse – sketches that have been hidden for over 200 years.

What's more, these hidden sketches were a continuation of the panoramic view from the previous page – so by digitally stitching two double page panoramas together, new complete views could been created that would never have been possible to see before – even by the artist himself!

Now, how could we display these new super long panoramas online whilst still allowing the detail to be seen?

The default width for our webpages is set at just under 1000 pixels across, this was just not enough to be able to show off these panoramas in any detail, so I decided that the easiest solution was to add scroll bars direct to the image, allowing them to be displayed across the page whilst at the same time allowing the complete panoramas to be studied in detail.

Cardiff 1678:

One of these newly generated images is of a panoramic view of Cardiff, containing an unique view of the medieval town as it was back in 1678.

To show this detailed sketch off in the best possible way, I decided to repurpose our interactive image navigator tool, which allows the user to pan around a high resolution image viewing details close up.

By using texts from a previously published article on medieval Cardiff, I was also able to pinpoint and highlight certain aspects of the panorama that were noteworthy – be it places that have remained unchanged since medieval times, or places that have long since vanished.

Francis Place goes global

To promote this work, the marketing team at the Museum distributed several Tweets and Facebook mentions. As well as being picked up by the BBC Wales news website and local media, we also published images onto the photo sharing website Flickr and added the extra information as notes embedded within the image. To make it a bit more user focused, I posted a comment asking users to guess where the artist was postitioned as he sketched… The foreground area of the sketch has altered so dramatically since 1678, it's not as easy as it seems….

Rhagor homepage featuring the Francis Place articles and interactives

Well, the iPad now has a release date for the US - early April, not mid-March as first expected. It will probably arrive in the UK late April. In the meantime we seem to have exhausted ourselves trying to decide whether or not it's a good thing - let alone a necessary thing. But then not being needed didn't make the iPod or iPhone any less desirable.

Apple's marketing for the iPad has taken a turn for the interesting, as their key words are "revolutionary" - possibly true - and "magical" - what?! Of course it's slick, sexy, a thing of beauty; there might well be something revolutionary about it; but - magical? Now using that concept to describe a piece of digital equipment, that's revolutionary! It's a piece of kit that lets us use our email and the web, look at our pics and videos and play with all our digital toys (150,000 of them apparently). Eventually we'll be able to use it to read e-books.

Previously Apple were promoting the iPad's similarity with the iPhone in terms of functionality, so that we'd all feel at home right away. Now, however, the iPad is revolutionary, magical and value for money. It sounds as though somewhere in the flurry of attention since its announcement, Apple have abandoned the "third category" concept that so many people questionned, and instead are positioning the iPad as a gem of a product, something lovely and affordable and just so much fun. A must-have accessory, perhaps. In which case, where does that leave the e-reading function? It was never primarily an e-book reader, more for all-round media consumption, but publishers were desperately looking forward to the healthy, straightforward supply deal offered by Apple, and any further delay in launching iBook in the UK is surely going to be a major cause for concern.