Secure Data Handling - Part 1 - Understanding Encoding

Have you ever entered a search term into a web browser, hit "Enter", and noticed that the URL changes to something like this?

https://example.com/search?query=hello%20world%21

The %20 and %21 represent a space () and an exclamation mark (!) respectively. They are part of URL encoding.

What is Encoding?

Encoding is the process of converting information into a different format, typically for storage or transmission.

A simple example is the "A1Z26" cipher, where each letter of the alphabet is replaced by its corresponding number (A=1, B=2, C=3, etc.). For instance, the word apple would be encoded as 1-16-16-12-5.

Practical Applications of Encoding

1. Base64 Encoding

Base64 encoding is one of the most common encoding schemes used to convert binary data into a text format. It’s often used to encode binary files (like images or attachments) so they can be transmitted over protocols that only support text (like HTTP, JSON, or email).

Example:
Plain Text: Hello, world!
Base64 Encoded: SGVsbG8sIHdvcmxkIQ==

Use cases:

• Encoding files for email attachments
• Embedding images directly into HTML or CSS

2. URL Encoding

URL encoding (also called "percent encoding") converts special characters into a safe format for URLs. For example, spaces and characters like &, =, and ? have special meanings in URLs, so they need to be encoded before transmission.

Example:
Plain Text: Hello world!
URL Encoded: Hello%20world%21

Use cases:

• Encoding user input for GET requests
• Encoding query strings or path parameters in URLs

3. HTML/XML Encoding

HTML or XML encoding ensures that special characters in data (like <, >, or &) are correctly interpreted by web browsers or other systems reading the data. For example, the character < in HTML needs to be encoded as < to avoid being mistaken for HTML tags.

Example:
Plain Text: x < 1
HTML Encoded: x < 1

This encoding is essential for displaying content within web pages without unexpected behavior.

Use cases:

• Escaping special characters in HTML or XML

Encoding is Not Security

It's important to note that encoding is not a security measure.

Encoding is reversible. When you encode a piece of data, anyone can decode it. For example, with Base64 encoding, the padding (==) clearly signals the encoding method. Even if you invent a new encoding scheme, malicious actors can often identify patterns and reverse the process.

When working with sensitive information like passwords or personal details, you'll need encryption or hashing, which are designed for data protection.

Summary

Encoding is a simple yet essential concept in modern computing. It ensures that data can be safely and correctly transmitted across systems that might have different ways of processing it.

In the next part of this series, we’ll dive into encryption, which goes a step further to protect your data during transmission. Stay tuned!