1. Common Formats of Digital Certificates
Digital certificates come in various formats, some of the common formats include:
-
X.509 Certificate:
-
X.509 is the most common standard for digital certificates, defining the format of public key certificates and related verification processes. X.509 certificates are usually encoded in DER or PEM format.
-
DER (Distinguished Encoding Rules):
-
DER is a binary encoding rule typically used to represent the binary form of X.509 certificates.
-
PEM (Privacy Enhanced Mail):
-
PEM is a text-based encoding format commonly used to transmit X.509 certificates over text protocols. PEM format can contain DER encoded or Base64 encoded data.
-
PKCS#12 / PFX (Personal Information Exchange):
-
PKCS#12 is a file format used for storing and transferring private keys, public keys, and certificate chains. PFX is a common extension for PKCS#12.
-
PKCS#7 / P7B:
-
PKCS#7 is a digital signature standard typically used to sign and verify data integrity. P7B is a common extension for PKCS#7, usually used to store certificate chains.
-
JKS (Java KeyStore):
-
JKS is the key store format used in Java to manage keys and certificates.
-
JCEKS (Java Cryptography Extension KeyStore):
-
JCEKS is a key store format in the Java Cryptography Extension that supports more advanced encryption algorithms.
-
AC (Attribute Certificate):
-
AC is an extension of the X.509 certificate used to include additional attribute information in an X.509 certificate.
2. Converting between CER and PEM Format Digital Certificates
(1) Converting from PEM to CER:
-
Obtain the PEM certificate file: First, you need to have a PEM format digital certificate file.
-
Convert the PEM certificate to CER format:
-
Use the appropriate tools or programming languages to read the part of the PEM certificate file excluding the —–BEGIN CERTIFICATE—– and —–END CERTIFICATE—– markers.
-
Decode the Base64 encoded data to binary data.
-
Save the resulting binary data as a file with a .cer extension to obtain the CER format certificate.
openssl x509 -outform der -in demo.pem -out demo.cer
-
openssl x509: Use the x509 subcommand in OpenSSL to operate on X.509 certificates.
-
-outform der: Specify the output format as DER encoding.
-
-in demo.pem: Specify the input file as demo.pem, which is the PEM format certificate file.
-
-out demo.cer: Specify the output file as demo.cer, which is the DER format certificate file.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 |
|
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 |
|
The converted cer format digital certificate is consistent with the cer format digital certificate exported from Charles.
(2) Converting from CER to PEM:
-
Obtain the CER certificate file: First, you need to have a CER format digital certificate file.
-
Convert the CER certificate to PEM format:
-
Use appropriate tools or programming languages to read the binary data of the CER certificate.
-
Base64 encode the binary data.
-
Add the —–BEGIN CERTIFICATE—– marker at the beginning of the encoded data and the —–END CERTIFICATE—– marker at the end.
-
Save this marked data as a file with a .pem extension to obtain the PEM format certificate.
openssl x509 -inform der -in demo.cer -out demo.pem
The converted pem format digital certificate is consistent with the pem format digital certificate exported from Charles.
3. Calculating the Hash Value of Digital Certificates
When installing certificates on mobile devices, several reasons may cause the certificate file name to change:
-
System-generated hash naming: In some cases, mobile devices may hash the installed certificate file and use the generated hash as the file name. This hash is usually a hash of the certificate content, used to uniquely identify the certificate.
-
Specific file name format required by the system: Some mobile devices may have specific requirements for certificate file names, needing to conform to a certain naming format. For example, in the Android system, the system requires the certificate file name to be in the format
.0, where is the hash value of the certificate content.
Therefore, we can change the file name on the phone to
For .pem certificate
mv cacert.pem `openssl x509 -inform PEM -subject_hash_old -in cacert.pem |head -1`'.0'For .cer certificate
mv cacert.cer `openssl x509 -inform DER -subject_hash_old -in cacert.cer |head -1`'.0'4. Certificate Loading Process and Principles
In the Android system, certificate management and parsing are handled by the Android security framework and Keystore system. Certificate management in the Android system mainly involves the following aspects:
-
Security Framework: The Android security framework provides a set of APIs and services for implementing various security features, including certificate management and usage. These functions include SSL/TLS connections, digital signatures, certificate verification, etc. Components in the security framework are responsible for parsing and processing certificate files, including PEM and DER format certificates.
-
Keystore System: Android’s Keystore is a secure repository for storing keys, certificates, and other sensitive information. Keystore provides a secure way to store and manage certificates to prevent the private keys of certificates from being accessed by malicious applications or attackers. The Keystore protects the private keys of certificates through system-level security protections, such as hardware-backed security elements (e.g., Trusted Execution Environment) or secure software containers (e.g., StrongBox).
-
System Certificate Store: The Android system also provides a system-level certificate store for storing trusted root certificates and other trusted certificates. These certificates are typically used to verify SSL/TLS connections or digital signatures. The Android system loads the system certificate store at startup and uses the certificates therein for system-level security features.
In the Android system, the parsing and use of certificates typically involve the following steps:
-
Loading Certificates: The Android system loads the root certificates and other trusted certificates from the system certificate store at startup. These certificates are stored in the system’s certificate store and managed by the system’s security framework and Keystore system.
-
KeyStore.getInstance(): Get the KeyStore instance for loading and managing certificates and keys.
-
KeyStore.load(): Load certificates and keys into the KeyStore from files or input streams.
-
CertificateFactory.getInstance(): Get the CertificateFactory instance for parsing certificates.
-
CertificateFactory.generateCertificate(): Generate certificate objects from input streams.
-
Resources.openRawResource(): Load resource files in Android applications, such as certificate files.
-
Parsing Certificates: When an application needs to use a certificate, the system calls the certificate parsing component in the security framework to parse the certificate file. These components can recognize the format of the certificate and extract key information such as public keys, issuer information, validity periods, etc.
-
X509Certificate class: Represents X.509 standard certificate objects, providing methods for obtaining certificate information such as public keys, issuer information, validity periods, etc.
-
CertificateFactory.generateCertificate(): Generate certificate objects from input streams, which can be used for parsing certificates.
-
Verifying Certificates: When establishing secure connections or verifying digital signatures, the system uses the information obtained from the parsed certificate to verify the validity of the certificate. This includes verifying the certificate chain, checking the validity period of the certificate, verifying the issuer’s signature, etc.
-
TrustManagerFactory.getInstance(): Get the TrustManagerFactory instance for creating TrustManager instances to verify server-side certificates.
-
KeyStore.getInstance(): Get the KeyStore instance for obtaining trusted root certificates for the system and applications.
-
SSLContext.getInstance(): Get the SSLContext instance for creating SSL/TLS connections.
-
SSLContext.init(): Initialize the SSLContext using TrustManager and KeyManager.
-
SSLSocketFactory.setHostnameVerifier(): Set the hostname verifier for verifying server hostnames.
-
Storing Certificates: For certificates or keys generated by applications, the system uses the Keystore system to securely store and manage them. The Keystore stores these certificates and keys in secure hardware or software containers to prevent unauthorized access by applications or attackers.
In summary, in the Android system, the management and use of certificates are jointly implemented by the security framework, Keystore system, and system certificate store. These components ensure the security and reliability of certificates and provide secure certificate management and usage functions at the system level. Therefore, when you place a .pem format certificate file in the system certificate directory, the system automatically converts it to a binary format .cer file for easier handling and use. This conversion is typically performed automatically by the system’s certificate management tools or certificate storage services, requiring no manual operation by users.
5. Practical Configuration of Proxies
(1) Installing PEM Certificates Using Mobile Applications
1. Select the cacert.pem digital certificate for installation
2. Name it pem
3. View the certificate
You will find that the file has been converted to a cer binary file digital certificate.
4. Export the file for comparison with the original cer digital certificate
The result shows that the installation of the pem format digital certificate, through the system’s certificate installation program, will convert the pem format digital certificate into a cer format digital certificate.
(2) User Certificate Packet Capture Testing
Packet capture effect:
(3) Transferring User Certificates to the System Certificate Directory
Use the Move_Certificates-v1.9 module to transfer certificates from the user certificate directory to the system certificate directory.
(4) System Certificate Packet Capture Testing
SSL certificate error, significantly better than the user certificate packet capture effect.
(5) Manually Pushing PEM Certificates to the System Directory
1. Push the pem certificate into the system directory
Push the certificate to the Move_Certificates module directory.
2. Change the permissions to 644
3. Restart the phone and wait for the Move_Certificates plugin to redirect the certificate to the system certificate directory
4. Packet capture effect
(6) Creating a PEM Format Digital Certificate Consistent with the Original System Certificate
1. Processing the source system certificate to PEM format digital certificate includes base64 and signature information
platina:/system/etc/security/cacerts # cat 7892ad52.0
-----BEGIN CERTIFICATE-----
MIIClDCCAhqgAwIBAgIILCmcWxbtBZUwCgYIKoZIzj0EAwIwfzELMAkGA1UEBhMC
VVMxDjAMBgNVBAgMBVRleGFzMRAwDgYDVQQHDAdIb3VzdG9uMRgwFgYDVQQKDA9T
U0wgQ29ycG9yYXRpb24xNDAyBgNVBAMMK1NTTC5jb20gRVYgUm9vdCBDZXJ0aWZp
Y2F0aW9uIEF1dGhvcml0eSBFQ0MwHhcNMTYwMjEyMTgxNTIzWhcNNDEwMjEyMTgx
NTIzWjB/MQswCQYDVQQGEwJVUzEOMAwGA1UECAwFVGV4YXMxEDAOBgNVBAcMB0hv
dXN0b24xGDAWBgNVBAoMD1NTTCBDb3Jwb3JhdGlvbjE0MDIGA1UEAwwrU1NMLmNv
bSBFViBSb290IENlcnRpZmljYXRpb24gQXV0aG9yaXR5IEVDQzB2MBAGByqGSM49
AgEGBSuBBAAiA2IABKoSR5CYG/vvw0AHgyBO8TCCogbR8pKGYfL2IWjKAMTH6kMA
VIbc/R/fALhBYlzccBYy3h+Z1MzFB8gIH2EWB1E9fVwHU+M1OIzfzZ/ZLg1Kthku
WnBaBu2+8KGwytAJKaNjMGEwHQYDVR0OBBYEFFvKXuXe0oGqzagtZFG22XKbl+ZP
MA8GA1UdEwEB/wQFMAMBAf8wHwYDVR0jBBgwFoAUW8pe5d7SgarNqC1kUbbZcpuX
5k8wDgYDVR0PAQH/BAQDAgGGMAoGCCqGSM49BAMCA2gAMGUCMQCK5kCJN+vp1RPZ
ytRrJPOwPYdGWBrssd9v+1a6cGvHOMzosYxPD/fxZ3YOg9AeUY8CMD32IygmTMZg
h5Mmm7I1HrrW9zzRHM76JTymGoEVW/MSD2zuZYrJh6j5B+BimoxcSg==
-----END CERTIFICATE-----
Certificate:
Data:
Version: 3 (0x2)
Serial Number: 3182246526754555285 (0x2c299c5b16ed0595)
Signature Algorithm: ecdsa-with-SHA256
Issuer: C=US, ST=Texas, L=Houston, O=SSL Corporation, CN=SSL.com EV Root Certification Authority ECC
Validity
Not Before: Feb 12 18:15:23 2016 GMT
Not After : Feb 12 18:15:23 2041 GMT
Subject: C=US, ST=Texas, L=Houston, O=SSL Corporation, CN=SSL.com EV Root Certification Authority ECC
Subject Public Key Info:
Public Key Algorithm: id-ecPublicKey
Public-Key: (384 bit)
pub:
04:aa:12:47:90:98:1b:fb:ef:c3:40:07:83:20:4e:
f1:30:82:a2:06:d1:f2:92:86:61:f2:f6:21:68:ca:
00:c4:c7:ea:43:00:54:86:dc:fd:1f:df:00:b8:41:
62:5c:dc:70:16:32:de:1f:99:d4:cc:c5:07:c8:08:
1f:61:16:07:51:3d:7d:5c:07:53:e3:35:38:8c:df:
cd:9f:d9:2e:0d:4a:b6:19:2e:5a:70:5a:06:ed:be:
f0:a1:b0:ca:d0:09:29
ASN1 OID: secp384r1
NIST CURVE: P-384
X509v3 extensions:
X509v3 Subject Key Identifier:
5B:CA:5E:E5:DE:D2:81:AA:CD:A8:2D:64:51:B6:D9:72:9B:97:E6:4F
X509v3 Basic Constraints: critical
CA:TRUE
X509v3 Authority Key Identifier:
keyid:5B:CA:5E:E5:DE:D2:81:AA:CD:A8:2D:64:51:B6:D9:72:9B:97:E6:4F
X509v3 Key Usage: critical
Digital Signature, Certificate Sign, CRL Sign
Signature Algorithm: ecdsa-with-SHA256
30:65:02:31:00:8a:e6:40:89:37:eb:e9:d5:13:d9:ca:d4:6b:
24:f3:b0:3d:87:46:58:1a:ec:b1:df:6f:fb:56:ba:70:6b:c7:
38:cc:e8:b1:8c:4f:0f:f7:f1:67:76:0e:83:d0:1e:51:8f:02:
30:3d:f6:23:28:26:4c:c6:60:87:93:26:9b:b2:35:1e:ba:d6:
f7:3c:d1:1c:ce:fa:25:3c:a6:1a:81:15:5b:f3:12:0f:6c:ee:
65:8a:c9:87:a8:f9:07:e0:62:9a:8c:5c:4a
SHA1 Fingerprint=4C:DD:51:A3:D1:F5:20:32:14:B0:C6:C5:32:23:03:91:C7:46:42:6D
platina:/system/etc/security/cacerts #2. Generate System Default Format Certificate File
For .pem certificate
openssl x509 -inform PEM -text -in cacert.pem > 7591945e.0For .cer certificate
openssl x509 -inform DER -text -in cacert.cer > 7591945e.03. Edit the generated file
Move the —–BEGIN CERTIFICATE—– to the beginning of the file.
-----BEGIN CERTIFICATE-----
MIIFPDCCBCSgAwIBAgIGAYyvnmk8MA0GCSqGSIb3DQEBCwUAMIGiMTMwMQYDVQQD
DCpDaGFybGVzIFByb3h5IENBICgyOCBEZWMgMjAyMywgT05MWVhJVS1QQykxJTAj
BgNVBAsMHGh0dHBzOi8vY2hhcmxlc3Byb3h5LmNvbS9zc2wxETAPBgNVBAoMCFhL
NzIgTHRkMREwDwYDVQQHDAhBdWNrbGFuZDERMA8GA1UECAwIQXVja2xhbmQxCzAJ
BgNVBAYTAk5aMB4XDTIzMTIyNzA4NTA0M1oXDTI0MTIyNjA4NTA0M1owgaIxMzAx
BgNVBAMMKkNoYXJsZXMgUHJveHkgQ0EgKDI4IERlYyAyMDIzLCBPTkxZWElVLVBD
KTElMCMGA1UECwwcaHR0cHM6Ly9jaGFybGVzcHJveHkuY29tL3NzbDERMA8GA1UE
CgwIWEs3MiBMdGQxETAPBgNVBAcMCEF1Y2tsYW5kMREwDwYDVQQIDAhBdWNrbGFu
ZDELMAkGA1UEBhMCTlowggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAwggEKAoIBAQCJ
TK1VSeT/k60mk8XIXLd/wSvl9XQpb6n2ouoybMj/vGSnr49reuzdZLAX2L4wR4E5
pIzCGGnYOKM0HgY8REiMPGRdPQujxkqlqlMj69/mQiUM6DNuP6QEIMaFqBD64pjd
Jv2X00FR/WTn+hulwA/iQyo43E9/lZ75yIa4eldEJKjLoteZAKrw6FAAEXec7Dfw
mGYvrVUBKBeQsHpSNo547hw1WStmfw7RbFgrOA2HPZb+3nbqAJb/zV2f9xfqRQZr
+H7GEzDv2jDvPGTKEYEe5BAy3xSxWiKsFGfmbJhebfQob08dh5PjJilB/ZZ8Phwe
3SYxpjZorgVmrKSPmdBnAgMBAAGjggF0MIIBcDAPBgNVHRMBAf8EBTADAQH/MIIB
LAYJYIZIAYb4QgENBIIBHROCARlUaGlzIFJvb3QgY2VydGlmaWNhdGUgd2FzIGdl
bmVyYXRlZCBieSBDaGFybGVzIFByb3h5IGZvciBTU0wgUHJveHlpbmcuIElmIHRo
aXMgY2VydGlmaWNhdGUgaXMgcGFydCBvZiBhIGNlcnRpZmljYXRlIGNoYWluLCB0
aGlzIG1lYW5zIHRoYXQgeW91J3JlIGJyb3dzaW5nIHRocm91Z2ggQ2hhcmxlcyBQ
cm94eSB3aXRoIFNTTCBQcm94eWluZyBlbmFibGVkIGZvciB0aGlzIHdlYnNpdGUu
IFBsZWFzZSBzZWUgaHR0cDovL2NoYXJsZXNwcm94eS5jb20vc3NsIGZvciBtb3Jl
IGluZm9ybWF0aW9uLjAOBgNVHQ8BAf8EBAMCAgQwHQYDVR0OBBYEFFMQZywS+KKZ
6b7kqWVDwcw/YbvWMA0GCSqGSIb3DQEBCwUAA4IBAQB9YIDwLnBTHYT8E7cZ1DCd
1scQRak/iZwbTbcmxPLT/Mi/FTKDRNFXvA/3JO/SOZZMGmRU1TGiU1IfMyIIf5YT
yz7iuv1MTs5G8jlJ2iguIsAu4keBda3y9gOyDjGfdya1IXI1BxJV9zjw+uKqQS7k
zP7PEeCsJN41OPy1mgNuCxsLC1EvvilvTH7Q4fJGTtu/ImIgdeyIThuEc6m/hswv
DjAfGw9LNBJJNgQqiKid/lBp68HEDVTrsjEtRc9wnfiqlKXQlZnnt17vV8I8Bxdp
+ihpOt44gfR79Z9qPuScDejMYb3vCySsbOApPlS327ty4Abf1dlxFBVsjWbmQoV0
-----END CERTIFICATE-----
Certificate:
Data:
Version: 3 (0x2)
Serial Number: 1703753443644 (0x18caf9e693c)
Signature Algorithm: sha256WithRSAEncryption
Issuer: CN = "Charles Proxy CA (28 Dec 2023, ONLYXIU-PC)", OU = https://charlesproxy.com/ssl, O = XK72 Ltd, L = Auckland, ST = Auckland, C = NZ
Validity
Not Before: Dec 27 08:50:43 2023 GMT
Not After : Dec 26 08:50:43 2024 GMT
Subject: CN = "Charles Proxy CA (28 Dec 2023, ONLYXIU-PC)", OU = https://charlesproxy.com/ssl, O = XK72 Ltd, L = Auckland, ST = Auckland, C = NZ
Subject Public Key Info:
Public Key Algorithm: rsaEncryption
Public-Key: (2048 bit)
Modulus:
00:89:4c:ad:55:49:e4:ff:93:ad:26:93:c5:c8:5c:
b7:7f:c1:2b:e5:f5:74:29:6f:a9:f6:a2:ea:32:6c:
c8:ff:bc:64:a7:af:8f:6b:7a:ec:dd:64:b0:17:d8:
be:30:47:81:39:a4:8c:c2:18:69:d8:38:a3:34:1e:
06:3c:44:48:8c:3c:64:5d:3d:0b:a3:c6:4a:a5:aa:
53:23:eb:df:e6:42:25:0c:e8:33:6e:3f:a4:04:20:
c6:85:a8:10:fa:e2:98:dd:26:fd:97:d3:41:51:fd:
64:e7:fa:1b:a5:c0:0f:e2:43:2a:38:dc:4f:7f:95:
9e:f9:c8:86:b8:7a:57:44:24:a8:cb:a2:d7:99:00:
aa:f0:e8:50:00:11:77:9c:ec:37:f0:98:66:2f:ad:
55:01:28:17:90:b0:7a:52:36:8e:78:ee:1c:35:59:
2b:66:7f:0e:d1:6c:58:2b:38:0d:87:3d:96:fe:de:
76:ea:00:96:ff:cd:5d:9f:f7:17:ea:45:06:6b:f8:
7e:c6:13:30:ef:da:30:ef:3c:64:ca:11:81:1e:e4:
10:32:df:14:b1:5a:22:ac:14:67:e6:6c:98:5e:6d:
f4:28:6f:4f:1d:87:93:e3:26:29:41:fd:96:7c:3e:
1c:1e:dd:26:31:a6:36:68:ae:05:66:ac:a4:8f:99:
d0:67
Exponent: 65537 (0x10001)
X509v3 extensions:
X509v3 Basic Constraints: critical
CA:TRUE
Netscape Comment:
....This Root certificate was generated by Charles Proxy for SSL Proxying. If this certificate is part of a certificate chain, this means that you're browsing through Charles Proxy with SSL Proxying enabled for this website. Please see http://charlesproxy.com/ssl for more information.
X509v3 Key Usage: critical
Certificate Sign
X509v3 Subject Key Identifier:
53:10:67:2C:12:F8:A2:99:E9:BE:E4:A9:65:43:C1:CC:3F:61:BB:D6
Signature Algorithm: sha256WithRSAEncryption
Signature Value:
7d:60:80:f0:2e:70:53:1d:84:fc:13:b7:19:d4:30:9d:d6:c7:
10:45:a9:3f:89:9c:1b:4d:b7:26:c4:f2:d3:fc:c8:bf:15:32:
83:44:d1:57:bc:0f:f7:24:ef:d2:39:96:4c:1a:64:54:d5:31:
a2:53:52:1f:33:22:08:7f:96:13:cb:3e:e2:ba:fd:4c:4e:ce:
46:f2:39:49:da:28:2e:22:c0:2e:e2:47:81:75:ad:f2:f6:03:
b2:0e:31:9f:77:26:b5:21:72:35:07:12:55:f7:38:f0:fa:e2:
aa:41:2e:e4:cc:fe:cf:11:e0:ac:24:de:35:38:fc:b5:9a:03:
6e:0b:1b:0b:0b:51:2f:be:29:6f:4c:7e:d0:e1:f2:46:4e:db:
bf:22:62:20:75:ec:88:4e:1b:84:73:a9:bf:86:cc:2f:0e:30:
1f:1b:0f:4b:34:12:49:36:04:2a:88:a8:9d:fe:50:69:eb:c1:
c4:0d:54:eb:b2:31:2d:45:cf:70:9d:f8:aa:94:a5:d0:95:99:
e7:b7:5e:ef:57:c2:3c:07:17:69:fa:28:69:3a:de:38:81:f4:
7b:f5:9f:6a:3e:e4:9c:0d:e8:cc:61:bd:ef:0b:24:ac:6c:e0:
29:3e:54:b7:db:bb:72:e0:06:df:d5:d9:71:14:15:6c:8d:66:
e6:42:85:74
4. Packet capture effect
6. Conclusion
In the Android system, certificate management and usage are jointly implemented by the security framework, Keystore system, and system certificate store. These components ensure the security and reliability of certificates and provide secure certificate management and usage functions at the system level. Therefore, when you place a .pem format certificate file in the system certificate directory, the system automatically converts it to a binary format .cer file for easier handling and use. This conversion is typically performed automatically by the system’s certificate management tools or certificate storage services, requiring no manual operation by users. Therefore, the packet capture effect configured using the above method is similar.
Source: Xuanxue Forum