How software helps and hinders official cover-ups

As was reported yesterday by the Georgia Straight, on September 17, the Vancouver Police Department’s professional standards section (PSS) dismissed a complaint leveled against a VPD officer by Audrey Siegl, a prominent First Nations activist. Siegl alleges the officer deliberately bumped into her during a February 13, 2015, protest. leaving her with a cut lip and a bruise.

As shown by the Straight, the dismissal was accompanied by a public copy of the PSS report into the complaint that was very heavily censored with many pages entirely blacked out.

One of the thoughts that I had when I saw all those black bars was to wonder exactly how the VPD went about its wholesale censorship.

A quick check of the Internet shows that for multi-page documents, there’s a good chance that the VPD, like thousands of government agencies and departments around the world, uses dedicated redaction software — software that often produces an Adobe Acrobat PDF file.

It turns out though, that an awful lot of the documents redacted using software are nowhere near as “sanitized” of their sensitive information as they appear to be or as government agencies would like. Very often government employees have used the wrong software or they’ve used the right software the wrong way, or…did I mention Adobe Acrobat?

The Vancouver Police Department’s professional standards section may have used actual redaction software to cleanly censor the Siegl complaint report but it’s easy enough to find documents released by other VPD sections that have been redacted with the software equivalent of crayons.

Nothing defeats security like human error

Last year, the VPD’s anti-graffiti squad released search warrants used on June 3, 2014, to raid an East Vancouver house, in connection with the “No Pipelines” graffiti vandalism.

Both the address of the house and the name of the person targeted by the raid were redacted (as one would expect) but the job was done in an unexpectedly sloppy way.

The results suggest that the physical warrant pages were digitally scanned as 1-bit, pure black and white images and then imported into an image editing program, where they were upsampled to 8-bit grayscale.

Editing makes at least part of the target’s three-word name legible.

Words were not redacted by drawing solid black rectangles over them but rather by trying to cross them out with multiple parallel horizontal lines, drawn with a fuzzy-edged, grayscale paintbrush-style tool.

The method left enough room for error that simple post processing in another image editing program recovered readable letters and words from at least two of the redactions.

How redaction works and how it often doesn’t

The practice of censoring official documents by whiting out (or more often) blacking out text in place so that recipients can see the location of deletions, is commonly called “redaction”, a form of what government circles refer to as “sanitization“.

Manually, words and lines can be made illegible by covering them with opaque paint or black ink but removable black tape produces the most professional-looking results.

In the absence of computer technology, redaction as we recognize it, requires the final step of of photographic reproduction — it is only ever a copy of the  marked- or taped-over document that is given to the public.

Computers and the Internet changed everything to do with government documents, from the expectation of the public to receive them through the government’s ability to release and censor them.

Almost all redaction is now done using computer software. Either within the word processing programs where documents are created, or via secondary programs designed specifically to prepare documents for public distribution.

No matter where it starts though, most redaction appears to result in an Adobe Portable Document Format (PDF) file; with PDFs effectively taking the place of photography or photocopying in the redaction process.

Once again “Adobe” and “security” are mutually exclusive

Before Microsoft’s circa-2006 redaction Add-In for Microsoft Office Word 2003, (or a later version for Word 2007/2010  written by a Microsoft engineer) it wasn’t unusual for people to do their redacting in Word by using black drawing shapes to hide parts of a document and then to export the Word documents as Adobe PDF files.

This was a bad idea. Although the black shapes couldn’t be moved, the text was still underneath them and was easily accessed by simply copying and pasting the contents of the PDF.

Curiously, for the longest time, there were no better redaction tools available in Adobe’s actual PDF authoring software, Acrobat Professional — a program heavily used to prepare government documents for public distribution.

Adobe Acrobat Professional 8, released in 2006, was the first version of Adobe’s PDF authoring software to have true built-in redaction tools. Until then, Adobe directed customers to use an Acrobat add-on by Appligent, called Redax to securely redact PDFs (Redax is still said to be a popular with government agencies).

The bottom line is that in thousands of redacted PDF files created using pre-2006 Adobe Acrobat authoring software, the redactions may (like the Word documents exported as PDFs) be black boxes sitting over the text, with the text still available to be copied out of the PDF and pasted and read in another text document.

A particularly notable example of this was the “improperly redacted” PDF of the U.S. Transportation Security Administration’s 2008 security screening manual, which hit the news with a bang on December 7, 2009.

This capped a run of high-profile redaction screw-ups involving PDF files and Adobe was moved to post a short page entitled “How to properly redact PDF files“.

Adobe warned users that simply changing the background color to match the font color (like so: This is the wrong way to redact text.), or covering the embedded text was not enough. What users had to do, explained Adobe, was buy Acrobat Professional 8!

How buggy old software is an enemy of oppression

The redaction landscape seems to have changed little in the last six years. In addition to Adobe Acrobat Professional, there are countless dedicated redaction programs  –often producing PDFs — that are competing for a piece of a very big pie. Consider that every branch and every department of every government in the world has to be able to redact documents for public release.

Unfortunately (for secretive authoritarian types, that is) the basic inertia of bureaucracy means that a large percentage of the redacted documents produced around the world will continue to be badly made using very old software.

And I’m not just talking about the tens of millions of government computers around the world that are still running the 14-year-old Windows XP operating system. Click the images to enlarge them.