Essays

2018/05/17

Digitaal Tijdperk

Analoog of digitaal?

Toen in het begin van deze eeuw de digitale camera's in opmars waren moest er een afweging gemaakt worden of ik verder wilde met fotorolletjes of zou overstappen naar een digitale camera. Mijn Nikon F-801s is een prima camera maar niet digitaal.

De ver doorontwikkelde kwaliteit van film leverde destijds nog betere resultaten dan digitale camera's. Daarnaast had ik in de loop der jaren al een behoorlijk archief met negatieven en dia's opgebouwd. Ook mijn ouders hadden nog een aardige stapel 120 en 35 mm negatieven liggen.

Scanner

Uiteindelijk maar besloten om een gulden middenweg te kiezen. Door de aanschaf van een high-end negatief scanner kon ik de F-801s langer blijven gebruiken en de beelden toch digitaal maken en op Internet gebruiken.

Na ontwikkeling van een fotorolletje via de fotospeciaalzaak werd onmiddelijk alles gescand op hoge resolutie. Toen begon ook de taak om alle oude negatieven te scannen. In de loop der jaren werd hieraan waar mogelijk tijd besteed.

Na meer dan 15 jaar, twee onderhouds beurten en een reparatie van de scanner is het dan eindelijk zover. Meer dan 15.000 negatieven en dia's gescand die bijna 600 GB aan data oplevert. Eindelijk volledig digitaal!

Analoog beeld materiaal in digitaal tijdperk

Hierdoor wordt het mogelijk om oude familie-foto's te delen.

Terug te kijken op foto opdrachten van de 1e fotocursus.

Oude portretten nogmaals gebruiken.

En recent analoog werk online te zetten.

Camera

Nu de kwaliteit van de huidige digitale camera's sterk is verbeterd en het nauwelijks meer mogelijk is om een fotorolletje ontwikkeld te krijgen, heeft de onvermijdelijke overstap naar een digitale camera enkele jaren geleden plaats gevonden. Ook omdat het publiceren van digitale foto's vele malen sneller is en je meteen resultaat kan controleren.

Read more in Blog

coolscan

2017/10/27

Problem with Tomcat, ImageIO and ServletOutputStream

During testing a Java Servlet on Tomcat that uses ImageIO and ServletOutputStream to display several guitar chord diagrams, I ran into a strange problem, that I can't explain or solve. The pattern used was found in several locations on the Internet. It creates a BufferedImage that is written to the ServletOutputStream:

// Set PNG content type
response.setContentType("image/png");
// Construct a diagram creator for the provided chord name
DiagramCreator diagramCreator = new DiagramCreator(chordName);
// Create the image of a chord diagram.
BufferedImage diagram = diagramCreator.createImage(width, height, notes);
// Write the image to the output stream
ServletOutputStream out = response.getOutputStream();
ImageIO.write(diagram, "PNG", out);
out.close();

As long as the servlet "diagram.png" is called only once per HTML page, there are no issues and the correct image is rendered on the page. The problem starts when multiple calls to the same servlet are made on the same page. Simplified HTML example:

<img src="diagram.png?n=Cmaj&notes=E4/1/0,G3/3/0,C3/5/3">
E4/1/0,G3/3/0,C3/5/3
<img src="diagram.png?n=Cmaj&notes=E4/1/0,C4/2/1,G3/3/0,C3/5/3">
E4/1/0,C4/2/1,G3/3/0,C3/5/3
<img src="diagram.png?n=Cmaj&notes=G4/1/3,E3/4/2,C3/5/3">
G4/1/3,E3/4/2,C3/5/3

The images are rendered in the HTML page, but repeatedly calling the same page will render different images each time. The text below the image should always match the text inside the image. The red circled ones don't. There the diagram image belongs to another chord pattern. So the wrong image is rendered, indicating to me that somewhere there is a buffering or thread-safety issue.

There is some debate if ImageIO is thread-safe. Also Tomcat is using a ServletOutputStream pool where a stream might be used simultaneously by multiple browser threads? Using .isReady() didn't provide a solution either. Whatever the root cause, it's blocking further development of the application :(

Reference material:

The dangers of Javas ImageIO
I'm using the Java ImageIO to dynamically serve images and get strange Exceptions from time to time. Is this a bug in Tomcat?
Java Code Examples for javax.servlet.ServletOutputStream
ServletOutputStream.isReady() returns true while it is not yet ready
Should I close the servlet outputstream?
temp imageio files in Tomcat gnerated by botBinFormat ?
Broken Pipe when writing bytes in ServletOutputStream
Problem with multiple images in a JSP (Connection reset by peer)

Read more in Blog

2017/01/15

Nikon Coolscan & Nikon Scan 4.0 on Windows 7/8

We now live in a digital age where every smart-phone has a camera and images can be posted on-line almost immediately. The older generation of photographers can still remember a time when photography involved loading film into a camera and having it developed. It usually toke several days before the photographs were ready.

Even if one has switched to digital photography since, large archives of images on film still exist. To allow these old images on film to become usable in the digital age, you need a way to scan them. Investing in a high-end film scanner (e.g Nikon Coolscan LS-8000 ED) made sense in the early days of digital cameras with low resolution. It allowed for extended use of investments in analog camera equipment and produced better results than the digital cameras at that time. A lot has changed since then and high-end cameras now offer resolution that rivals film.

On the downside, computer hardware and operating systems have become obsolete making it harder to use this scanning equipment. E.g. the Nikon Scan driver and software were not updated to work after Windows XP. Even parts are getting scares and it might not be very long before Nikon stops servicing Nikon Coolscans.

A while ago I found an article "Nikon Coolscan and Nikon Scan 4.0: Driver for Windows Vista 64 bit, Windows 7 64 bit and Windows 8 64 bit" online, that describes how get the Nikon Scan 4.0 driver working on Windows 7/8. After applying the steps I can now use the Nikon Coolscan and Nikon Scan 4.0 on Windows 7, giving it a new lease on live.

Reference material:

Nikon Coolscan and Nikon Scan 4.0: Driver for Windows Vista 64 bit, Windows 7 64 bit and Windows 8 64 bit
http://imaging.nikon.com/lineup/scanner/scoolscan_8000_ed/
https://en.wikipedia.org/wiki/Nikon#Film_scanners
http://www.didero.nl/index.php/nikon-coolscans
https://petapixel.com/2015/04/24/12-reasons-photographers-still-choose-to-shoot-film-over-digital/
https://petapixel.com/2016/08/19/film-photography-making-stunning-comeback/
http://blog.controlspace.org/2010/05/nikon-scan-on-windows-7-and-vista-64.html

Read more in Blog

coolscan

2016/06/19

Koken CMS on HTTPS

Disclaimers

THIS HOWTO IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THIS HOWTO OR THE USE OR OTHER DEALINGS IN THIS HOWTO.

Netgear states: "Access by SSH is not discouraged, but is recommended for advanced users only. As such, using SSH is at the user's own risk."

This is how it worked in the situation of the baseline described below. Use it at your own risk.

Improvements

Please post concerns, comments and improvements for this blog on related Netgear Community "How to run Koken on https?" post to help improve it.

Background

The Koken Content Management System that is available as a ReadyNAS App from the RN Admin Pages' Apps section, will be installed to run on HTTP by default. It features a password protected Koken Admin Page.

Koken also boasts an optional plugin to allow for password protected private albums and a Cart plugin that can be used to sell photos.

It is never a good idea to send password or purchase information as cleartext (unencrypted) over HTTP. It has long since become common practice on the Internet to use HTTPS for such sites. Therefore you want to use HTTPS for a Koken driven website. Self-signed certificates cause browsers to issue warnings or disallow access to the site, so they cannot be used. You need certificates from a Certificate Authority (CA). Letsencrypt is a CA that offers a free, automated and open way to get certificates that you need to enable HTTPS.

Baseline

RN312 or RN516
ReadyNAS OS 6.5.0, 6.5.1 or 6.5.2 (Debian GNU/Linux 7 (wheezy))
Still works after upgrade to 6.6.0, 6.6.1, 6.7.1, 6.7.4, 6.7.5, 6.8.0, 6.8.1, 6.9.0, 6.9.1, 6.9.2 and 6.9.3 (Debian GNU/Linux 8 (jessie))

(Note: this how-to might also work on other RN models or OS versions, but that has not been tested.)

Prerequisites

MySQL and Koken installed on ReadyNas OS 6.5.0, 6.5.1, 6.5.2, 6.6.0, 6.6.1, 6.7.1, 6.7.4, 6.7.5, 6.8.0, 6.8.1, 6.9.0, 6.9.1, 6.9.2, 6.9.3.
Koken driven website accessible via a domain name from the Internet.
ReadyNAS needs outgoing Internet access to connect to Letsencrypt, retrieve downloads for the installation and periodically renew the certificate.

Preparation

You need to have a domain name connected to your Koken public web pages as the certificate process needs these names as input. If you don't have that yet, work on that first.

In this howto "yourdomain.com" and "www.yourdomain.com" need to be replaced by your own domain name. Koken (installed from RN Admin Page) runs on port 7100 by default. Make sure you have port forwarding in place on your router that point the outside port 80 to 7100 internally and outside port 443 to 7443 internally. (7443 will be added to the Apache configuration later.)

Next find your Linux and Apache version, as it influences which script and what plugins you can use from Letsencrypt and CertBot. And also which certificate files you need to add in the Apache configuration later.

Apache version

apachectl -V

Server version: Apache/2.2.31 (Debian)

These are the certificate files you will add later to your Apache config.

cert.pem
Server certificate only. This is what Apache < 2.4.8 needs for SSLCertificateFile.
chain.pem
All certificates that need to be served by the browser excluding server certificate, i.e. root and intermediate certificates only. This is what Apache < 2.4.8 needs for SSLCertificateChainFile, and what nginx >= 1.3.7 needs for ssl_trusted_certificate.

Linux version

cat /etc/*-release

PRETTY_NAME="ReadyNASOS 6.5.0"
NAME="Debian GNU/Linux"
VERSION_ID="7"
VERSION="7 (wheezy)"
ID=debian
ANSI_COLOR="1;31"
HOME_URL="http://www.debian.org/"
SUPPORT_URL="http://www.debian.org/support/"
BUG_REPORT_URL="http://bugs.debian.org/"

The OS is Debian GNU/Linux 7 (wheezy). Therefore you cannot use the "certbot" command, but have to use certbot-auto instead.

Since OS 6.5.0 features Apache 2.2, you cannot use the CertBot Apache plugin. This howto uses the webroot plugin instead.

(Note: OS 6.6.0 and upwards features Debian GNU/Linux 8 (jessie). You may now be able to use the "certbot" command, but that has not been tested.)

Create letsencrypt dir

Create a letsencrypt dir where we can store the cerbot script. In this howto we use /home/letsencrypt.

mdkir /home/letsencrypt

Install certbot-auto

cd /home/letsencrypt
wget https://dl.eff.org/certbot-auto

--2016-06-18 10:02:12--  https://dl.eff.org/certbot-auto
Resolving dl.eff.org (dl.eff.org)... 173.239.79.196
Connecting to dl.eff.org (dl.eff.org)|173.239.79.196|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 44115 (43K) [text/plain]
Saving to: ‘certbot-auto’
certbot-auto                         100%[=====================================================================>]  43.08K   286KB/s   in 0.2s
2016-06-18 10:02:14 (286 KB/s) - ‘certbot-auto’ saved [44115/44115]

Allow the downloaded script to be executable.

chmod a+x certbot-auto

Run the script to install all the needed components (e.g. python).

/home/letsencrypt/certbot-auto

Bootstrapping dependencies for Debian-based OSes...
Hit http://security.debian.org wheezy/updates InRelease
Hit http://security.debian.org wheezy/updates/main amd64 Packages
Ign http://apt.readynas.com 6.5.0 InRelease
Hit http://egnyte-cdn.egnyte.com 6.5 InRelease
Hit http://egnyte-cdn.egnyte.com 6.5/egnyte amd64 Packages
Ign http://mirrors.kernel.org wheezy InRelease 
... (lots of other lines) ...
Setting up python-pkg-resources (0.6.24-1) ...
Setting up python-setuptools (0.6.24-1) ...
Setting up python-virtualenv (1.7.1.2-2) ...
Processing triggers for libc-bin ...
Creating virtual environment...
Installing Python packages...
Installation succeeded.

During this process you might be asked to input an email address and agree to the license in a full terminal screen with blue background. Fill in your certificate administrator email address and select OK. Select Agree, if you want to accept the license.

After seeing the messages "Installation succeeded", you should be able to create the certificate.

Create certificate

/home/letsencrypt/certbot-auto certonly --webroot -w /data/koken/web -d yourdomain.com -d www.yourdomain.com

IMPORTANT NOTES:
 - Congratulations! Your certificate and chain have been saved at
   /etc/letsencrypt/live/yourdomain.com/fullchain.pem. Your cert
   will expire on 2016-09-15. To obtain a new or tweaked version of
   this certificate in the future, simply run certbot-auto again. To
   non-interactively renew *all* of your certificates, run
   "certbot-auto renew"
 - If you lose your account credentials, you can recover through
   e-mails sent to someone@yourdomain.com.
 - Your account credentials have been saved in your Certbot
   configuration directory at /etc/letsencrypt. You should make a
   secure backup of this folder now. This configuration directory will
   also contain certificates and private keys obtained by Certbot so
   making regular backups of this folder is ideal.
 - If you like Certbot, please consider supporting our work by:
Donating to ISRG / Let's Encrypt:   https://letsencrypt.org/donate
Donating to EFF:                    https://eff.org/donate-le

The created certificates can be found in: /etc/letsencrypt/live/yourdomain.com. Note the symoblic links pointing to the latest version in ../../archive.

cd /etc/letsencrypt/live/yourdomain.com
ls -al

total 16
drwxr-xr-x 1 root root 82 Jun 17 17:24 .
drwx------ 1 root root 28 Jun 17 17:24 ..
lrwxrwxrwx 1 root root 38 Jun 17 17:24 cert.pem -> ../../archive/yourdomain.com/cert1.pem
lrwxrwxrwx 1 root root 39 Jun 17 17:24 chain.pem -> ../../archive/yourdomain.com/chain1.pem
lrwxrwxrwx 1 root root 43 Jun 17 17:24 fullchain.pem -> ../../archive/yourdomain.com/fullchain1.pem
lrwxrwxrwx 1 root root 41 Jun 17 17:24 privkey.pem -> ../../archive/yourdomain.com/privkey1.pem

You are now ready to start using the certificate in your Apache web server.

Apache configuration

The Apache config for koken can be found in /apps/koken/http.conf. First create a backup so you have a working configuration to fall back on.

cp /apps/koken/http.conf /apps/koken/http.conf.ORIGINAL

Now add an HTTPS listener to http.conf. Copy the VirtualHost block for *:7100 to *:7443. Add a Rewrite rule to the *:7100 VirtualHost that will redirect all requests on HTTP to HTTPS. Add the SSL options, SSL Engine and SSL certificates to the *:7443 block.

vi /apps/koken/http.conf

Listen 7100
Listen 7443 https
<VirtualHost *:7100>
        ServerAdmin admin@localhost
DocumentRoot /apps/koken/web
        <Directory />
                Options FollowSymLinks
                AllowOverride None
        </Directory>
        <Directory /apps/koken/web/>
                Options Indexes FollowSymLinks MultiViews
                AllowOverride None
                Order allow,deny
                allow from all
        </Directory>
        RewriteEngine on
        ReWriteCond %{SERVER_PORT} !^443$
        RewriteRule ^/(.*) https://%{HTTP_HOST}/$1 [NC,R,L]
        ErrorLog /apps/koken/error.log
        LogLevel warn
</VirtualHost>
<IfModule mod_ssl.c>
<VirtualHost *:7443>
        ServerAdmin admin@localhost
        DocumentRoot /apps/koken/web
        <Directory />
                Options FollowSymLinks
                AllowOverride None
        </Directory>
        <Directory /apps/koken/web/>
                Options Indexes FollowSymLinks MultiViews
                SSLOptions +StdEnvVars
                AllowOverride None
                Order allow,deny
                allow from all
        </Directory>
        ErrorLog /apps/koken/error.log
        LogLevel warn
        SSLEngine on
        SSLCertificateFile "/etc/letsencrypt/live/yourdomain.com/cert.pem"
        SSLCertificateKeyFile "/etc/letsencrypt/live/yourdomain.com/privkey.pem"
        SSLCertificateChainFile "/etc/letsencrypt/live/yourdomain.com/chain.pem"
</VirtualHost>
</IfModule>

Restart your Apache server

service apache2 restart

Connect your browser to your domain on HTTP. It should now redirect to HTTPS and the browser should not complain about missing or misconfigured certificates any more.

Renewal

The certificates are valid for 90 days. You may want to automate renewal. Manual renewal can be achieved with

/home/letsencrypt/certbot-auto renew

-------------------------------------------------------------------------------
Processing /etc/letsencrypt/renewal/yourdomain.com.conf
-------------------------------------------------------------------------------
The following certs are not due for renewal yet:
  /etc/letsencrypt/live/yourdomain.com/fullchain.pem (skipped)
No renewals were attempted.

Add the following to your crontab to run renewal once each day. (Letsencrypts recommends 2 a day). In the example, the renewal runs each day at 01:47 AM (local time) and appends the output to a log file. Letsencrypt recommends using a random value for the minute parameter. Probally to spread load for renewal requests on Letsencrypt servers.

crontab -e

47 01 * * * /home/letsencrypt/certbot-auto renew >> /home/letsencrypt/certbot.log

Reference material

https://letsencrypt.org/getting-started/
https://certbot.eff.org/docs/
http://stackoverflow.com/questions/21415181/can-i-stop-the-this-website-does-not-supply-identity-information-message
http://help.koken.me/customer/portal/questions/16177649-change-entire-site-to-https

Read more in Blog