Vigenere Cipher — Kasiski / Index-of-Coincidence Key-Length Recovery

Polyalphabetic substitution: each plaintext letter is shifted by the letter of a repeating keyword (c_i = p_i + k_{i mod L} mod 26); it breaks because the key repeats, not because the shift itself is weak.

The signal that betrays it

- Ciphertext is letters-only, roughly the same length as the plaintext, and its overall letter distribution is *flatter than English but not uniform* — a raw frequency count won't spike the way a Caesar/Substitution & Shift Cipher — recovered by letter-frequency, not by algebra ciphertext does. That flattening (average index of coincidence around 0.04-0.05, versus ~0.0667 for English and ~0.0385 for random 26-letter text) is the tell that a single shift got averaged out over several alphabets. - Look for repeated ciphertext fragments of 3+ characters recurring at regular distances — a repeated plaintext trigraph re-aligning with the same key offset produces the same ciphertext trigraph again. This is the classic Kasiski tell, and it only shows up because the key repeats. - Challenge framing that says "Vigenere", "polyalphabetic", or "repeating key" is a giveaway, but don't stop there — check that the key is genuinely *shorter* than the message and *reused*. If key length is close to or exceeds the message length, you're no longer looking at a breakable Vigenere — see "What does not work" below.

How to exploit

1. Kasiski examination — get candidate key lengths. Find every repeated substring of length ≥3 in the ciphertext, record the distance between each pair of occurrences, and take the GCD / common factors across several such distances. The most frequent common factor is the likely key length L. Corroborate with more than one repeated sequence before committing — a single repeat can be coincidental. - Ciphertext-length rule of thumb we rely on: under ~100 characters Kasiski is unreliable (too few repeats to trust); 100-200 characters gives a tentative estimate; 200+ characters is reliable with multiple corroborating repeats; 1000+ is near-certain. 2. Index of coincidence — confirm/refine L. For each candidate L, split the ciphertext into L interleaved columns (position i → column i mod L) and compute the average index of coincidence across columns: IC = Σ n_i(n_i - 1) / (N(N-1)), n_i = count of letter i in the column, N = column length. The correct L pushes the average IC up toward the English value (~0.0667); wrong lengths sit near the random-text value (~0.0385, sometimes reported ~0.04-0.05 for real Vigenere ciphertext due to averaging/small-sample noise). Use this to break ties or extend Kasiski's guess when repeats are scarce. 3. Per-column shift recovery. Once L is trusted, every column is now a plain shift (Substitution & Shift Cipher — recovered by letter-frequency, not by algebra) cipher — 26 possible offsets. Score each offset with a chi-squared fit against expected English letter frequencies (χ² = Σ (O_i - E_i)² / E_i) rather than raw frequency-peak matching; chi-squared is markedly more robust on the short samples you get per column. The offset minimizing χ² is the key letter for that column. 4. Recombine and sanity-check. Concatenate the per-column shifts into the keyword, decrypt the full ciphertext, and verify it reads as real language (don't trust the statistics alone on a short or noisy ciphertext — confirm the output). 5. Recognize the structural sibling. This is the exact same "find period L, transpose into L columns, solve each column as a single-alphabet cipher" skeleton as Repeating-Key XOR — recovering key length and key from a Vigenère-style stream (repeating-key XOR over bytes) — that page is the byte/XOR analog of the same idea. The two diverge only in how L is found: XOR uses Hamming-distance minimization between blocks (because equal key bytes XOR to low bit-density), Vigenere uses Kasiski repeats / index of coincidence (because equal key letters realign plaintext letter statistics). If a challenge hands you raw bytes or hex instead of an alphabetic ciphertext, you're almost certainly looking at the XOR sibling, not literal Vigenere — check the operation (mod-26 addition vs. XOR) and alphabet size before picking which recovery method to run.

What does not work

- Running frequency analysis straight on the raw ciphertext, the way you would for a plain substitution or shift cipher, fails outright — that's the entire point of a polyalphabetic cipher: it averages the letter distribution across L different shifts and flattens the single peak a monoalphabetic cipher would show. You must split into columns by the correct L first; frequency analysis only works *after* that split. - Trusting a single repeated trigraph as your key-length evidence. Short or unlucky ciphertexts produce coincidental repeats — a plaintext sequence colliding with a different key alignment can accidentally reproduce the same ciphertext fragment. Always look for the common factor across *several* repeat-distances, or cross-check with the index-of-coincidence scan, before committing to an L. - Attempting Kasiski/IC on short ciphertext (well under ~100 characters): there just aren't enough repeats or enough column data for either statistic to be reliable — the method is data-hungry by construction. - Assuming the technique still applies when the key is as long as (or longer than) the message. If the key never actually repeats over the ciphertext, there is no periodic structure for Kasiski to find and every candidate L's index of coincidence will look equally "random." That is not this cipher failing to break — it means you are effectively facing a one-time-pad-style construction instead, which needs a different attack path (key/plaintext reuse elsewhere), not a longer Kasiski search. - Trusting the raw frequency-peak match per column on very short columns (long key relative to ciphertext length) even after L is correctly identified — too few samples per column make simple frequency matching noisy; chi-squared scoring against expected English frequencies degrades more gracefully and is the more reliable choice here.

Related: Substitution & Shift Cipher — recovered by letter-frequency, not by algebra, Affine Cipher — a tiny 312-key space that known-plaintext or brute force both break instantly, Repeating-Key XOR — recovering key length and key from a Vigenère-style stream, Cryptanalysis Methodology — classify the family before you reach for an attack.