Cracking the code: The continuing intrigue surrounding the HMAS Sydney

Peter Hore

This is a transcript of a public lecture given by Peter Hore at the National Archives of Australia on 27 July 2007.

Thank you very much indeed, it's a great pleasure to be here tonight, and I appreciate so many people have turned out on a Friday night in Canberra to hear what I might have to say. I'll move about a bit, so you can see me at some stage.

My name is Peter Hore; I come from England. I assume, from the distant shores of England, that everybody knows the tale of HMAS Sydney, but I met somebody this afternoon who didn't. So, I must take a minute or two just to reprise the story.

Sydney was the pride of the Australian fleet in 1940–1941. In July of 1940, Sydney had taken on two Italian cruisers northwest of Crete in the Mediterranean and she'd sunk one of those ships and she'd damaged the other and driven it off.

She had played a prominent part under Admiral Cunningham in the British Mediterranean fleet, and Andrew Cunningham himself was extremely sorry to see Sydney leave at Christmas 1941 and return to Australia.

Sydney returned to Australia; there was a short change of officers and she then undertook some patrol and convoy duties in the Indian Ocean and around Australia. And in November, 1941, she was returning from a convoy to the Sunda Straits. She was on route to Fremantle, when she disappeared.

Her disappearance and the subsequent defence was a great shock to the Australian Navy and to the Australian nation. It is not just one of the greatest tragedies and mysteries of the sea of the Second World War, but it is also an unfinished chapter in Australian history, which I hope in my small way I can help write the ending to.

What happened afterwards is that the German survivors from the battle – there were no Australian survivors – the German survivors were picked up at sea. Some of them had also reached the coast to Red Bluff, north of Carnarvon, and they were taken to a prisoner of war camp.

Their captain was amongst the prisoners of war. His name was Theodor Detmers, and he spent from 1941 until 1946 in a prisoner-of-war camp in Australia. And in the winter of '46–47, he was repatriated in a steamship called the Arantes to Cookshaven in Germany.

So, this is the story, however, of how Theodor Detmers tried to report to the authorities at home in Germany what had happened during the battle.

Tom Frame, writing in 1989, said that Theodor told what happened – I'm sorry, I must go back one moment. In January of 1945, Detmers and some of his fellow prisoners were so bored that even though they were told the war was over in Europe, they dug a tunnel out of their prison camp and they escaped.

Detmers was at liberty for about, I think, 20 days before he was recaptured. And Tom Frame – Bishop Frame – says in his book about the raider Kormoran, when he was writing in 1989, that when Detmers was recaptured after his escape from prisoner-of-war camp, a coded diary was found in his possession. Neither Australian nor British analysts to whom it was subsequently passed could decipher the code.

This short statement, I'm afraid, really contains two errors: Detmers was not carrying a diary, but he was carrying a notebook. And we know that by 1945 – that's the notebook and that's at least the way I first saw the notebook, it has seven or eight pages that just look like that, and I will show you a larger version in a moment – and by 1945, the Australian authorities had in fact decoded that diary. And I'm making no claims to have been the first person to have done so. And that is how I first saw that account, heavily blacked out. And again, I will show you an uncensored version, both of which come from the archives here.

Wes Olsen, who wrote another excellent book about this subject in 2000, says that when Detmers was searched after his recapture, he was found to be carrying a German/English dictionary. And when it was discovered that small dots had been placed under certain letters, the dictionary was confiscated and sent to a secret cryptanalysis unit in Melbourne, a unit called FRUMEL, for analysis. It was subsequently established that Detmers had used the Vigenere Grid cipher to record two accounts of the action between Sydney and Kormoran.

This is closer to the truth, but it's not true. The dictionary exists, that's the title page from the dictionary, but it is in two volumes. It weighs about, I should think, seven or eight pounds. It's not the sort of thing if you were going walkabout in the Australian bush, you'd want to carry with you.

In fact, he'd left the dictionary behind in prison camp, amongst the many possessions which he managed to accumulate. And after a spell in solitary confinement as a punishment for having escaped – and in hospital because he'd had a heart attack, poor man – he was reunited with his possessions and he was allowed to keep the dictionary, which returned with him in the Arantes when he was repatriated in the winter of 1946–47.

This dictionary is now in the possession of Detmers' nephew in Hamburg. It was published by Cassell in 1939. It's in good condition: it's tightly bound, it's not stained by sea water, as you might expect if it has been in the lifeboat with Detmers after the battle, or in the lifeboat when he was swamped when he was being towed by the steamer Centaur. So, it seems like the dictionary was given to him in prisoner-of-war camp by his captors.

According to the nephew, when he was a boy his uncle had shown him the dictionary and explained how he had deceived his guards by hiding his battle report in the pages. And again, according to his nephew, he said an 'American woman', but we know that was Barbara Winters, came to visit his widowed aunt in the early 1990s and asked to see the dictionary.

Barbara Winters only had a short while available in Hamburg, so the nephew copied the pages. The book itself, the dictionary, was still with the nephew when his aunt went into a nursing home. So, the provenance of this book is good, and we know how it came to be in the nephew's possession.

This is the book, which the nephew showed me in Hamburg in 2003. Instead, Detmers did escape... sorry, I'll show you that too, because this is the teaser for you. This is one of the pages from the dictionary with the hidden account in it. And you can step up if you want to, but you're seeing it at several times life size. It's just slightly larger than A3 in size, or smaller than A4 in real size.

There is, as I said, a notebook, and this is what was captured on Detmers. The original notebook has been lost, along with apparently many original German documents. I don't quite understand this, because the English translations are in the archives, some of the original German documents are here, but others, I don't know what happened to them.

The copy of the notebook as it existed in the National Archives at the beginning of 2000, was also missing a page, it was missing page one. And page one of the notebook contains all of the essential navigational information that we wanted to look at. And it was the search for this page which eventually led us to a cache of documents in Australia.

Here, I have to tell you, as my friend David Mearns, who works for a company called Blue Water Recoveries, who is one of the best wreck hunters in the world. And quite by chance, I only met him a short time ago, two or three years ago. David turns out to be my neighbor in Sussex in England.

David persuaded the owner of this cache of documents to bring them to the National Archives. And if you can imagine an archivist being excited, then you can imagine Steve Stuckey – I think he has left you now hasn't he – Steve Stuckey was certainly very excited.

He sent us almost daily emails about the progress of this lady's arrival here in the archives with these documents. And if I read out one of his emails to you he says, 'The nearest and the best thing and the only copy in Australia, as far as I can tell, and the lady agrees that such an important document must be held in the archives'.

So, they were copied, the original went back to the owner. You can see these documents; they made far better copies than this here. And I think, when I came in tonight, somebody was looking at them on the screen outside.

So, I say the reference to the coded dictionary was wrong. There was this notebook, and Steve Stuckey from here got very excited about it. I think, that's the end of a happy story and the documents do reside now where they should do.

As I have stressed before, this notebook was in fact decoded in Australia in 1945. And as far as one can get excited about this subject, just this afternoon I was told by somebody at another conference where I'd been to, how it was decoded and the methodology that was used. I have to do more research from that, but that is another exciting find, which has happened in the last six hours since I saw you this morning. Fantastic.

My task, however, was to decrypt this notebook. I was aware from the early work I had done that the 1945 decryption contained errors of decryption, translation and transcription. That is what page two of the dictionary looks like. And I'm going to show you... I'll take just a few minutes to show you how we started the process of decryption.

The first thing to notice about this dictionary is that the author has used 30 letters in the alphabet. In addition to the normal letters we'd expect, there are four more letters, which he's invented. You see this? A-underline, B-underline, C-underline, and D-underline.

So, we know something immediately about the code. Code makers normally write grids. We know that this grid is 30 characters in direction up or down. There are some more – I'm just showing you one page here – but there are some characteristic entries. There's this funny squiggle here, which appears in this dictionary, and it's usually followed by a four-letter group.

If you look at those four-letter groups, the highest letter in any of those groups is K. K, that's the 10th letter. And it's not too difficult to work out, if you're used to this kind of thing, that that's a substitution table for numbers one to nine and zero.

And the next thing to notice from inspection of this document is that this little group of letters here, seven letters, occurs several times in the account. Now, the only reason that can be is because the author had started the decryption table anew each time.

So, that is what this squiggle must mean. Each time he had a break or a rest, or he wanted to write something new, he would put one of these squiggles in and – actually, it's very bad practice if you are using an encryption table – he started it over again.

So, we've got some important clues here. I say, this little group of letters here and these groups of four letters, which mean something to us. Now, I understand from what I've learned this afternoon that the original decoders used a frequency table. That's how you break tables normally.

You know, I suppose, that E-T-I-O-N-A-S – in that order – are the most common letters in the English language, and that pretty much applies to other languages too. So, if you know that, you can work out with a frequency table what E probably is, and that's how you start to break codes. That, and with a great deal of guessing.

This seven-letter group here must mean something that he's repeated. It's not 'Kormoron', because it's too long. It's not 'Sydney', because it's too short. And after a little bit of playing around, a little bit of guess and God, actually it's kreuzer, the German word for cruiser.

So, we have an entry into the key, into this table. And I'll just summarize what we've been through there. It's a bit like being at school, isn't it? So, it's a 30-letter code, 26 letters plus these four new letters, which the author had invented. He started a new line of code with this four-letter group, so we can guess, actually, that this is the date or time. I don't know what it is yet.

Also, there's also this seven-letter group here, which must be special, and we've guessed – I've jumped a slide here – but we've guessed it's kreuzer or cruiser. And, as I say, this very bad practice, this is clearly not somebody who was skilled or whose normal work it was to write code.

And you remember the place of those letters in the alphabet, if the first four are numbers in a substitution code then the fifth letter – and if it is cruiser – then the fifth letter is kruezer, is K. The sixth is R, the seventh, eighth, ninth, tenth, eleventh. We have seven letters – seven to eleven – of the code.

Now, we put them into a table like this, remember we said there is the alphabet he used, 30 letters. I don't know how wide the table is at the moment, I have no idea, but we're guessing now – and you can make lots of other assumptions and I did make other assumptions – but here's N as the fifth letter. I can't see it quite from my angle. And so on across the table.

If we go to the next table, let's not forget those. As I said, those are the funny four-letter groups whose highest letter is K. And it usually starts with H or I, which might be an indicator to us. If you're writing the time, especially if it's in a four-letter format, it starts with zero or one or possibly two. If you're writing a date, it starts with one, two or three.

So, I'm going to take a deep breath here and I'm going to assume that H means one and it's the first letter in our decryption table. And I'm going to put that in the decryption table. We'll just jump over that. I'm going to put it in here, into my decryption table. Again, it's a guess. I could be wrong, it could be two, it could be zero. But, I'm getting somewhere, I feel, with this code, particularly when I've spent hours over it and I've only got 45 minutes to tell you about it.

So, if I fill in those letters down the alphabet like that... OK. It's a substitution table; clearly it is a substitution table. So, we fill in the alphabet down here, not forgetting these four letters. I did the first time around. I was very short. Not forgetting the extra four letters.

And we're looking for a key word, just like when you ring up your credit card and they ask for you mother's maiden name, we're looking for a key word. And the only thing that looks vaguely familiar to me is this little piece of text here. We know it's German, of course. And we are looking for this key word in this decryption table.

Oops, I think, I've hidden something for you. So, I'm guessing, because I know what I'm looking for now. I'm dealing with his battle summary. And this word down here – again, it's a bit like doing a crossword puzzle – this word is gefechtsbericht, which is the German word for 'battle summary.'

If I assume that at the bottom here, and fill in the rest of this table, that's what the decryption table looks like. We knew that is was 30 letters in one direction, we now know he's used a 15–letter word along the bottom there conveniently. I've got the complete decryption table, and I'm now able to go back to that notebook and decrypt it right from the beginning, remembering that every time the author made one of those squiggles, I've got to start again. He could have made it considerably more difficult for me, had he not forgotten to do that.

I suspect that the author was tired or weak or whatever, because he used I and J interchangeably. They look very much the same in the dictionary. And later on, I'm going to also show you where he used L for I, because in that little dictionary, with the ink barely separated, they do look very much alike.

And do you remember those four-letter groups? I have assumed that that was one. And if I'm correct, here's the substitution table, substitution within the code: one to nine plus zero. Because he'd use I and J interchangeably, I've got to remember, he actually used one, two, three, four, five, six, seven, eight, nine, zero. And later on, when he was feeling, obviously, a bit tired at doing this, he used L as well. But that was a later piece of work, which puzzled me for a little while and caused me some difficulty.

That is how I first saw the 1945 work, which had been done in Melbourne. And this is how I saw it in 2003, 2004, when I asked for a new copy from the National Archives of Australia. I hope you can see that clearly. I don't know why they blanked it out in 1945, when they first released it. It wasn't particularly special or difficult code to break. But actually, the coders have made an error in this letter here as well. Can you see it?

There's a substitution table for the numerals. A-1, B-2, C-3, D-4, E-5, F-6, H – and he's missed out a letter. What happened to G? And he's skipped a number here, which has caused a number of people rather a lot of difficulty over the years.

We are very helped here because there was another document which we found. Archivists, and even the best of archivists, if they're keen on their subject, they squirrel things away or they leave them on a desk for a long time. I'm sure they don't do it at the National Archives of Australia. [laughs]

But, we found another cache of documents in the Admiralty Library in London. They were uncatalogued. The worst thing about an archivist: he files people's particulars under the color of their hair or their eyes rather than their surname. But, if he actually hasn't even filed them at all – ie they're uncatalogued – you don't know where on earth they are.

We found another set of documents. And it's a long story, but this document, when they came home in the passenger ship Orontes, in '46–47, the Germans typed up a new version of their battle summary. And we found this document, and we were able then to check what we had done against this new document. It looks very similar to some of the others.

And now we'll come to the dictionary, because that's what we're faced with. The dictionary almost caused me a divorce, because you need fresh eyes, you need good light, and you need a great deal of patience to detect this.

Can anybody see the dots [laughs] from this? Look at that piece of text there. And I'll blow it up for you. And I think, now, you can see the dots. You can see the first four letters: B-F-A-K. And you can see some more down here. So, we've got something out of this. Very exciting.

I did this on a Friday night. I'd been saving it up, because I knew how difficult it was going to be. I was saving it up. And I sat down on a Friday night, in my flat, looking out over the lights of London to do this work. And I got as far as 'B-F-A-K', and I thought, 'Oh, no. Here we go again. Another code'.

And this is what we got out of that: B-F-A-K. And that went on to do the next bit. And if you look at the next lot of dots, there they are. And I wrote those down. And that's what I got. Coders like five-letter groups. That's how they write things. There's nothing special about that.

And then, I did the next list of dots. And that's what I got out of it. Looks like a code, doesn't it? But, I recognized some words there from the work I've already done. I've got one of the four-letter groups, which I'm familiar with: 'Cumberland, N-Klassen, nach Westen'. He's spelled 'nach' very strangely, but I've got German text.

Is Steven Buss here? Steven? That's the surprise in the dictionary. It's not in code; it's in plain language.

I discovered this at about midnight UK time. And it was too early to ring anybody up in Australia, and all my friends in England had gone to bed. [laughs] But, I was very excited, because suddenly I had a plain-language text. And that is what we've gone on to do. We've gone to de-dot this dictionary. The account runs to several pages. But here, we have what we believe to be the ground zero, probably the first record that Detmers made of his battle summary.

Excuse me while I take some water.

So, we now have the account, which FRUMEL made in 1945. We have the account from Orontes, which I showed you very briefly. We have the account from the dictionary, which we've extracted from here. And we have the account from the fresh decoding, which I made of the notebook.

And I want to show you something else now, because that's what, actually, we get out of this dictionary. That's the substitution code, 26-10. And if you're familiar with the story, the 26th of October was the date that the German supply ship Cumberland departed from the Kormoran in the Indian Ocean.

And this is a comparison of the text. This is the FRUMEL transcript and translation. This is what Wes Olsen published in his book on the Sydney. This is my decrypt of this one, here. This is what the dictionary says. And this is what the Orontes typescript says.

I'm just showing you a portion of these accounts. The letters in red show you where the differences are. And there are important differences in meaning. Let's take the first one. It's because of that mistake in the substitution table, the FRUMEL account said '800 meters'. This, 'HM', means hectometers. That was a German unit of gunnery measurement. Artillery officers in Germany measure things in units of 100 yards.

So, the FRUMEL account said '800 yards'. Somebody, at some later stage, realized a mistake had been made, and they changed it to 9. But, they added a zero, which gives us, in some accounts, the fact that Sydney was at 9000 yards. She was never at 9000 yards – meters. I beg your pardon. I'm still in Imperial. She was always at 900 meters, and that's what all the other accounts tell us. This is a mistake, introduced in the process of transcription, in Melbourne, sometime in 1945. And this is a mistake because the original decoders didn't understand, for whatever reason, made a mistake with their substitution table.

This is a puzzle. This has puzzled many people. What on earth did this mean? But actually, if you know now what I've told you about the substitution table, that is 'east 2-6 south'. That's how that breaks down. And that's how it appears in the other accounts, right across the board there.

There's one strangeness here in the dictionary, he's spelled the name 'Malagga' instead of 'Malakka'. And I think, if you're saying 'Straat Malakka' in German, that sounds pretty similar.

And there's another oddity about the translation. The translation, the earlier versions of this, it cuts across. The German says, 'mitlaufen', which means 'runs alongside'. And I've translated it as 'takes station'. So, the cruiser took station on the starboard beam at 900 meters. There's no doubt about what the German tells us in these extracts.

And then, there's one more issue, which has puzzled researchers for many years. In some accounts, it says that 'the cruiser stops engines'. What it actually says in the original account, on the Melbourne version, it says, 'stops engines'. And somebody later has added, in brackets, 'cruiser stops engines'.

It says, 'stops engines' in the decrypt, but in the dictionary, it clearly says, 'the aircraft motor off'. And if you look in the Orontes typescript, it says, 'enemy turns the aircraft engine off'. So, this reference to Sydney stopping is another error, a misunderstanding, which has crept in over the years.

There is some corroboratory evidence here about this. This is the book, which was published in 1959, which was alleged to be Detmers' account, his own memoir of the cruise of the Kormoran. This is the German book, published in the same year.

I can only assume – the translator of this book was a man called Fitzgerald – he must have been in a devil's own rush to get the translation of this book out in the same year, because the Fitzgerald version of the book is 206 pages long, whereas the Detmers' original book is... oh, [laughs] I beg your pardon. My own transcription error here. The Detmers' version of the book is 263 pages long. It's nearly a fifth longer. There's much more in the German book by Detmers than there is in the English translation.

And there are also some important errors in it. And one of the errors – you remember this business about whether Sydney had stopped or not? Fitzgerald says, 'My eyes were glued to the bearing compass. And as soon as I saw the enemy had come practically to a standstill, I gave the order to de-camouflage'.

This is what he says in German, and that actually translates – and there's a big difference here in the meaning – that 'I stayed, fixed by the bearing compass, until the bearing of the enemy had steadied. And then, when the expected situation had arrived, I gave the order to de-camouflage'. In other words, Detmers is steaming along in one direction, and when the bearing steadies, Sydney is alongside him as well – mitlaufen, they're running alongside, together. That's what it says.

So, there we go. We can eliminate many of the misunderstandings that have crept in over the years in our analysis of what happened between these two ships.

Downstairs, in the case, I saw one of your documents from the archives, dated the 25th of November, which says that 'this information about the sinking of Sydney' – as it was known in Melbourne – 'must be kept most secret, as the enemy does not know'. Unfortunately, the enemy did know.

This is an extract, not from Kormoran. This is an extract from the German Kreistag's book, The German War Diary, which was kept in Germany. You may know the story. The British captured the entire archive in 1945, at Castle Tambach in southern Germany. And we took it back to Britain, where it stayed until the 1960s, when we gave it back to the Germans, having photocopied it, or put it all onto microfiche.

And this is talking about a radio message, from an unknown steamer who is being asked by radio from Australia about details of the engagement and the name of the ship from which the survivors were rescued. And it says here, 'The context of the message is not known'. And it's signed off by Grand Admiral Raeder.

And there in the next day is actually an intercept, an Admiralty signal. We know where the corresponding message is in the National Archives of Australia, and in the British Archives. They had intercepted a message, which is talking about Sydney's demise. They've got the name of the ship now. And this is Raeder. This is the 26th of November. The Germans knew. They knew before most people in Australia did that the Sydney had been sunk.

With all of this information... oh, let me just show this one, too. The British Navy, right after any engagement or campaign, writes a naval staff summary, and so do the Germans. This is a translation of the German staff summary of the cruise of the Kormoran.

Detmers had been sending his own war diary home in instalments. The last instalment he sent home on the 26th of October 1941, in the supply ship Cumberland. He'd made all of these copies in an effort to get his version of the story back to his own headquarters in Germany. And in 1943, Siebelt Habben, who was the ship's doctor, the second doctor and the acting dentist in the ship – can you imagine going to sea with an acting dentist? – he was repatriated under a prison-of-war scheme.

I find that incredible to start with. I didn't realize that, even at that stage of the war, we were talking to the Germans. But, there was a prisoner-of-war exchange scheme. Habben went to Germany. And a man called James Young, who had been captured by the Kormoran – I forget which ship – came back from Germany in the same exchange scheme.

And here it is. This is a translation of the German war staff diary. I don't know whether there's a copy here, but certainly there are various copies in England.

The German naval war staff received the first report of a battle on the 24th. They intercepted a signal on the 26th, which was deciphered on the 30th. So, the Germans were deciphering our signals.

This is the message, the text of the signal, which I know what that is in English, our outgoing version. And then, Habben, when Detmers discovered that Habben was going to be repatriated, the evidence they have from Habben's son, and from Messerschmidt, who was Detmers' adjutant in prison camp, was that Detmers made the doctor come to him and he made him learn the battle report by rote, so that when Habben got back to Germany, he was able to tell the story. And this battle summary was published in Germany, top secret, in December 1943.

With all of this information – including, I have to say, the astronautic navigation, because, according to Detmers, he turned into the sun to run away from the Sydney. And we can see that even that information, his course 250, checks with the actual information.

With all that information, we have one final test of proof, and that is to plot it on a chart. That's what it looks like on a chart, when we first did our plotting.

We're now able to say, with all of this information, what the shape of the battle was, what the relative positions of the ships were during the engagement. We can backtrack to what the noon position was because we know what time the meridian was on the 19th of November.

My colleague, David Mearns... anyway, this is a professional discussion we're having. We have a slight disagreement about where, now, the Kormoran is. And the positional information which we have is 26° 34" south, 111° east. Is it the action position, or is it the abandon ship position? Because after they had abandoned ship, Kormoran would have drifted off to the north, slowly, at perhaps one knot, one mile per hour.

We don't know that. But, it doesn't really matter because the scale of this is here. This is only about five miles. And if you draw an ellipsis around here, that's the area where we think, where we recommend – we strongly recommend – we should search for the Kormoran.

The next question, the last question, of course, is where is the Sydney? And we don't know. But, she's somewhere down there to the southeast, depending on how long she survives. If we can establish a datum here, we can find the Sydney. That's it. Thank you.

Collection references

Copyright National Archives of Australia 2017