Detailed Explanation of Unicode and ISO 10646 in C++

Relationship between Unicode and ISO 10646

Background Introduction

Unicode and ISO 10646 were originally two independent standards, but they have now been largely unified:

  • Unicode: Maintained by the Unicode Consortium, it includes character encoding, properties, processing rules, etc.
  • ISO 10646: Maintained by ISO, primarily defines character sets and encoding space.
  • Cooperation: They began working together in 1991 to ensure consistency in character encoding.

Core Concepts

  • Code Point: A unique numeric identifier for each character, such as <span>U+222B</span>
  • Encoding Space: Ranges from <span>U+0000</span> to <span>U+10FFFF</span>, totaling 1,114,112 code points.
  • Plane: Divides the encoding space into 17 planes, each containing 65,536 characters.

Unicode Support in C++

Universal Character Name

C++ uses <span>\u</span> and <span>\U</span> escape sequences to represent Unicode characters:

#include <iostream>
#include <string>
#include <locale>

using namespace std;

// Example 1: Basic Unicode Character Representation
void basicUnicodeExamples() {
    cout << "=== Basic Unicode Examples ===" << endl;
    
    // Using \u escape sequence (4-digit hexadecimal)
    cout << "Greek Alpha: \u0391" << endl;        // Α
    cout << "Greek Beta: \u0392" << endl;         // Β
    cout << "Mathematical Integral: \u222B" << endl; // ∫
    
    // Using \U escape sequence (8-digit hexadecimal)
    cout << "Smiling Face: \U0001F600" << endl;   // 😀
    cout << "Chinese: \U00004E2D\U00006587" << endl; // 中文
}

// Example 2: Multilingual Text Processing
void multilingualText() {
    cout << "\n=== Multilingual Text ===" << endl;
    
    // European Languages
    cout << "French: Caf\u00E9 au lait" << endl;        // Café
    cout << "German: Stra\u00DFe" << endl;              // Straße
    cout << "Spanish: Ni\u00F1o" << endl;               // Niño
    
    // Greek
    cout << "Greek: \u0395\u03C5\u03C7\u03B1\u03C1\u03B9\u03C3\u03C4\u03CE" << endl; // Ευχαριστώ
    
    // Russian
    cout << "Russian: \u0417\u0434\u0440\u0430\u0432\u0441\u0442\u0432\u0443\u0439" << endl; // Здравствуй
    
    // Arabic
    cout << "Arabic: \u0627\u0644\u0633\u0644\u0627\u0645 \u0639\u0644\u064A\u0643\u0645" << endl; // السلام عليكم
    
    // Japanese
    cout << "Japanese: \u3053\u3093\u306B\u3061\u306F" << endl; // こんにちは
    
    // Chinese
    cout << "Chinese: \u6B22\u8FCE\u5B66\u4E60C++" << endl; // 欢迎学习C++
}

// Example 3: Mathematical Symbols and Special Characters
void mathematicalSymbols() {
    cout << "\n=== Mathematical Symbols ===" << endl;
    
    // Basic Mathematical Symbols
    cout << "Pi: \u03C0" << endl;                    // π
    cout << "Sum: \u2211" << endl;                   // ∑
    cout << "Infinity: \u221E" << endl;              // ∞
    cout << "Square Root: \u221A" << endl;           // √
    cout << "Integral: \u222B" << endl;              // ∫
    cout << "Partial Differential: \u2202" << endl;  // ∂
    
    // Relational Operators
    cout << "Not Equal: \u2260" << endl;             // ≠
    cout << "Less or Equal: \u2264" << endl;         // ≤
    cout << "Greater or Equal: \u2265" << endl;      // ≥
    
    // Arrows
    cout << "Arrows: \u2190 \u2192 \u2191 \u2193" << endl; // ← → ↑ ↓
    cout << "Double Arrows: \u21D0 \u21D2" << endl;  // ⇐ ⇒
}

// Example 4: Currency Symbols
void currencySymbols() {
    cout << "\n=== Currency Symbols ===" << endl;
    
    cout << "Dollar: $" << endl;
    cout << "Euro: \u20AC" << endl;                  // €
    cout << "Pound: \u00A3" << endl;                 // £
    cout << "Yen: \u00A5" << endl;                   // ¥
    cout << "Rupee: \u20B9" << endl;                 // ₹
    cout << "Bitcoin: \u20BF" << endl;               // ₿
}

// Example 5: Using Unicode in Identifiers
class InternationalCalculator {
private:
    double \u03C0; // Using Greek letter π as variable name
    double \u221A; // Using square root symbol as variable name
    
public:
    InternationalCalculator() : \u03C0(3.141592653589793), \u221A(1.41421356237) {}
    
    // Method name containing Unicode character
    double calculate\u03C0Value(double radius) {
        return \u03C0 * radius * radius;
    }
    
    double calculate\u221AValue() {
        return \u221A;
    }
    
    void display\u03C0() {
        cout << "The value of \u03C0 is: " << \u03C0 << endl;
    }
};

// Example 6: Unicode String Operations
#include <codecvt>
#include <locale>
void unicodeStringOperations() {
    cout << "\n=== Unicode String Operations ===" << endl;
    
    // UTF-8 String
    string utf8_str = "Hello 世界 \u03A9 \u20AC";
    cout << "UTF-8 String: " << utf8_str << endl;
    cout << "UTF-8 String length (bytes): " << utf8_str.length() << endl;
    
    // Wide String
    wstring wide_str = L"Hello 世界 \u03A9 \u20AC";
    wcout << L"Wide String: " << wide_str << endl;
    wcout << L"Wide String length (characters): " << wide_str.length() << endl;
    
    // C++11 u8 prefix
    const char* utf8_literal = u8"UTF-8 literal: 中文 \u0393 \u20BD";
    cout << utf8_literal << endl;
}

// Example 7: Unicode Character Property Check
#include <cctype>
void unicodeCharacterProperties() {
    cout << "\n=== Unicode Character Properties ===" << endl;
    
    // Check character type (Note: Standard library mainly handles basic character set)
    char basic_char = 'A';
    cout << "Is 'A' alphabetic? " << isalpha(basic_char) << endl;
    cout << "Is 'A' uppercase? " << isupper(basic_char) << endl;
    
    // For Unicode characters, more complex libraries are needed (like ICU)
    // Here we only demonstrate the basic concept
    wchar_t greek_alpha = L'\u0391';
    wcout << L"Greek Alpha is wide character: " << greek_alpha << endl;
}

// Example 8: File Encoding Operations
#include <fstream>
void unicodeFileOperations() {
    cout << "\n=== Unicode File Operations ===" << endl;
    
    // Write to UTF-8 file
    ofstream outfile("unicode_example.txt");
    if (outfile.is_open()) {
        outfile << u8"Multilingual Text:\n";
        outfile << u8"English: Hello World\n";
        outfile << u8"French: Bonjour le monde\n";
        outfile << u8"German: Hallo Welt\n";
        outfile << u8"Chinese: \u4F60\u597D\u4E16\u754C\n";
        outfile << u8"Japanese: \u3053\u3093\u306B\u3061\u306F\u4E16\u754C\n";
        outfile << u8"Mathematical: \u03C0 \u221A \u222B \u2211\n";
        outfile.close();
        cout << "Unicode file created successfully!" << endl;
    }
    
    // Read file (ensure console supports UTF-8)
    ifstream infile("unicode_example.txt");
    if (infile.is_open()) {
        string line;
        cout << "File contents:" << endl;
        while (getline(infile, line)) {
            cout << line << endl;
        }
        infile.close();
    }
}

// Example 9: Code Point Conversion
void codePointConversion() {
    cout << "\n=== Code Point Conversion ===" << endl;
    
    // Display code point information for some characters
    struct UnicodeChar {
        const char* name;
        const char* utf8;
        uint32_t code_point;
    };
    
    UnicodeChar chars[] = {
        {"Latin A", "A", 0x0041},
        {"Greek Alpha", "\u0391", 0x0391},
        {"Chinese '中'", "\u4E2D", 0x4E2D},
        {"Integral", "\u222B", 0x222B},
        {"Smile", "\U0001F600", 0x1F600}
    };
    
    for (const auto& uc : chars) {
        cout << uc.name << ": " << uc.utf8 
             << " -> U+" << hex << uppercase << uc.code_point 
             << dec << endl;
    }
}

int main() {
    // Set locale to support Unicode output
    setlocale(LC_ALL, "en_US.UTF-8");
    
    cout << "=== C++ Unicode and ISO 10646 Demonstration ===" << endl;
    cout << "Unicode Version: Supports up to 149,813 characters" << endl;
    cout << "ISO 10646: Universal Character Set (UCS)" << endl;
    cout << "==============================================" << endl;
    
    basicUnicodeExamples();
    multilingualText();
    mathematicalSymbols();
    currencySymbols();
    
    // Test International Calculator
    InternationalCalculator calc;
    calc.display\u03C0();
    cout << "Area of circle with radius 5: " << calc.calculate\u03C0Value(5.0) << endl;
    cout << "Square root of 2: " << calc.calculate\u221AValue() << endl;
    
    unicodeStringOperations();
    unicodeCharacterProperties();
    unicodeFileOperations();
    codePointConversion();
    
    cout << "\n=== Demonstration Complete ===" << endl;
    return 0;
}

Compilation and Running Instructions

Compilation Command:

# Linux/Mac (requires UTF-8 support)
g++ -std=c++11 -o unicode_demo unicode_demo.cpp
./unicode_demo

# Windows (using a terminal that supports UTF-8)
g++ -std=c++11 -o unicode_demo.exe unicode_demo.cpp
unicode_demo.exe

Unicode Encoding Forms

Comparison of UTF-8, UTF-16, and UTF-32

#include <iostream>
#include <string>
using namespace std;

void encodingComparison() {
    cout << "=== Unicode Encoding Forms ===" << endl;
    
    const char* chinese = "中"; // U+4E2D
    
    cout << "Character: " << chinese << endl;
    cout << "UTF-8 encoding (hex): ";
    for (int i = 0; chinese[i] != '\0'; i++) {
        printf("%02X ", (unsigned char)chinese[i]);
    }
    cout << endl;
    
    // UTF-8: Variable-length encoding (1-4 bytes)
    // UTF-16: 2 or 4 bytes
    // UTF-32: Fixed 4 bytes
    
    cout << "UTF-8 advantages:" << endl;
    cout << "- ASCII compatible" << endl;
    cout << "- Space efficient for Latin text" << endl;
    cout << "- Self-synchronizing" << endl;
}

Summary of Important Concepts

  1. Code Point: Numeric identifier for Unicode characters.
  2. Encoding Forms: UTF-8, UTF-16, UTF-32.
  3. Planes: 17 planes, with the Basic Multilingual Plane (BMP) being the most commonly used.
  4. C++ Support: Universal character names, wide characters, Unicode literals.
  5. Compatibility: ASCII is a subset of Unicode.

This example comprehensively demonstrates the support for Unicode and ISO 10646 in C++, including character representation, string processing, file operations, and other practical application scenarios.

Detailed Explanation of Unicode and ISO 10646 in C++

Related posts

  1. The Booming Development of Vibration Sensors: Tronics Joins the Market
  2. Firmware Extraction Without Chip Removal in Automotive Applications
  3. Analysts Claim Rubin Chip to Tape Out by Year-End, NVIDIA Achieves Unmatched Momentum
  4. The Urgency of Domesticating Pressure Sensors in China (Part 2)
  5. Eight-Layer PCB Fabrication | High-Speed Signal Integrity and Impedance Control
  6. Zhenlei Technology: The King of ADC/DAC Chips
  7. Overview of C++ Character Sets
  8. Detailed Explanation of User-Defined Functions in C++
  9. Understanding the C++ Standard Library Type: string
  10. C++ Object-Oriented Programming: From Built-in Types to Custom Types
  11. Detailed Explanation of Using Directive in Multi-Function C++ Programs
  12. Detailed Explanation of C++ Program Structure and Basic Concepts
  13. Choosing Between C++ and Java: A Comprehensive Guide for Computer Science Students on Employment, Development, and Core Competencies
  14. cpp-httplib: A Minimalist HTTP/HTTPS Server and Client Library in C++
  15. Some Tips for C/C++ Programming
  16. Summary of C++ Interview Questions from Major Companies in 2025
  17. Zserio: An Open Source Library Based on C++
  18. Frontline | Gold Medal Classroom: Luo Xianjin’s ‘C++ Programming’ Makes Students the Main Characters
  19. Adjusting Learning Strategies for C++ Competitive Programming Challenges

Leave a Comment