Notebook

文件名规范引发的 iTunes 备份还原错误及解决方案

@@ Tags: iTunes;备份;还原;错误修复
@@ Date: 2025-02-24

用户还原之前的iTunes备份时, 出现如下错误:

<plist version="1.0">
<array>
	<string>DLMessageProcessMessage</string>
	<dict>
		<key>ErrorCode</key>
		<integer>2</integer>
		<key>ErrorDescription</key>
		<string>_restoreRegularFiles:size: rename error: No such file or directory (2) at path "/private/var/mobile/.backup.i/var/mobile/Containers/Data/Application/9B671B4F-ED81-43E1-BD67-258DA97116B9/Documents/Audio books/Starwars/4 New Republic Era (5 to 25 ABY)/22 The Jedi Academy Trilogyèampions of the Force/01 Champions Of The Force.mp3" (MBErrorDomain/2)</string>
		<key>MessageName</key>
		<string>Response</string>
	</dict>
</array>
</plist>

用户的环境如下:

备份: iPhone 13 Pro Max(iPhone14,3) - iOS 15.4
目标: iPhone 13 Pro Max(iPhone14,3) - iOS 18.3 and 18.3.1
系统: Windows 11 Pro x64, 10.0.26100
iTunes 版本: iTunesMobileDevice.dll(1190.100.1.2)

分析错误原因

经检查 Manifest.db 文件，发现清单数据库中并未存在报错项目的上级目录:

错误项目信息:
- domain: AppDomain-org.videolan.vlc-ios
- relativePath: Documents/Audio books/Starwars/4 New Republic Era (5 to 25 ABY)/22 The Jedi Academy Trilogyèampions of the Force/01 Champions Of The Force.mp3
- fileId: ddab7b0a154cbc9b934b80331ced23dd820ffd0f
不存在的上级项目:
- domain: AppDomain-org.videolan.vlc-ios
- relativePath: Documents/Audio books/Starwars/4 New Republic Era (5 to 25 ABY)/22 The Jedi Academy Trilogyèampions of the Force
- fileId: 5da63f6ebaf0b256a7a1a654daaeadec91fdd920
但存在类似的上级项目:
- fileId: b8ae2d19ce590a182d23561dded5dc34f053e42e
- domain: AppDomain-org.videolan.vlc-ios
- relativePath: Documents/Audio books/Starwars/4 New Republic Era (5 to 25 ABY)/22 The Jedi Academy Trilogyèampions of the Force

fid_ddab7b0a = 'Documents/Audio books/Starwars/4 New Republic Era (5 to 25 ABY)/22 The Jedi Academy Trilogyèampions of the Force/01 Champions Of The Force.mp3'
fid_b8ae2d19 = 'Documents/Audio books/Starwars/4 New Republic Era (5 to 25 ABY)/22 The Jedi Academy Trilogyèampions of the Force'

left = fid_ddab7b0a[:len(fid_b8ae2d19) + 1]
right = fid_b8ae2d19
print('Left       : ', left)
print('Right      : ', right)
print('Compare    : ', left == right)
print('LeftEncode : ', left.encode('utf-8'))
print('RightEncode: ', right.encode('utf-8'))

index = left.index('è')
leftChar = left[index:index+2]
rightChar = right[index]
leftBytes = leftChar.encode('utf-16-be')
rightBytes= rightChar.encode('utf-16-be')
print('leftChar   :  {}={: <16}, hex({})'.format(leftChar, str(leftBytes), leftBytes.hex(' ')))
print('rightChar  :  {}={: <16}, hex({})'.format(rightChar, str(rightBytes), rightBytes.hex(' ')))

输出:

Left       :  Documents/Audio books/Starwars/4 New Republic Era (5 to 25 ABY)/22 The Jedi Academy Trilogyèampions of the Force
Right      :  Documents/Audio books/Starwars/4 New Republic Era (5 to 25 ABY)/22 The Jedi Academy Trilogyèampions of the Force
Compare    :  False
LeftEncode :  b'Documents/Audio books/Starwars/4 New Republic Era (5 to 25 ABY)/22 The Jedi Academy Trilogye\xcc\x80ampions of the Force'
RightEncode:  b'Documents/Audio books/Starwars/4 New Republic Era (5 to 25 ABY)/22 The Jedi Academy Trilogy\xc3\xa8ampions of the Force'
leftChar   :  è=b'\x00e\x03\x00', hex(00 65 03 00)
rightChar  :  è=b'\x00\xe8'     , hex(00 e8)

关键在于 Trilogye\xcc\x80ampions 与 Trilogy\xc3\xa8ampions, 前者 e\xcc\x80 后者 \xc3\xa8(在UTF-8 编码下)。

这其实是一个Unicode 中的字素分解的现象，前者是该字符在 Unicode Normalization 是采用 NFD 规范下产生的编码 è，后者是在 NFC 规范下产生的编码 &#xe8;

NFC（Normalization Form Canonical Composition）又称完全组合

NFC 规范，将字符组合为标准的组合形式。

NFC 规范，也是大多数平台的文件系统采用的归一化规范，如：Windows 和大多数 Linux 发行版。

示例: 신(신)

NFD（Normalization Form Canonical Decomposition）又称完全分解

NFD 规范，它将字符分解为基本字符和组合字符。

NFD 规范，是 macOS、iOS 的 HFS+ 文件系统采用的归一化规范[2] [3]。

示例: 신(신)

可以看到在完全分解下 `e 字符被拆解成了两个字符: e 和 ̀

根据以上信息，可以推论这大概是 iOS 15.4 的一个Bug，在备份时，将两个包含 è 路径采用了两个不同的 Unicode Normalization 规范来编码。从而导致在iOS 18.3.1 还原时，两个字符被解码成两个不同的路径，导致还原失败。

修复问题

在尝试修复问题之前，我们先对清单中的所有数据进行一些分析，判断问题规模，并查找具体原因。

import sqlite3
import hashlib

class ManifestDB:
    def __init__(self,):
        self.file = None
        self.conn = None
        self.cursor = None
        
    def open(self, file):
        self.file = file
        self.conn = sqlite3.connect(file)
        self.cursor = self.conn.cursor()

    def close(self):
        self.conn.close()

    @staticmethod
    def fileId(domain, relativePath):
        return hashlib.sha1(f"{domain}-{relativePath}".encode('utf-8')).hexdigest()
    
    def exists(self, fileID):
        self.cursor.execute("SELECT COUNT(*) FROM Files WHERE fileID = ?", (fileID,))
        return self.cursor.fetchone()[0] > 0

    def files(self):
        self.cursor.execute("SELECT fileID, domain, relativePath, flags, file FROM Files")
        return self.cursor.fetchall()

import unicodedata
import plistlib

def NormalizationForm(string):
    nfc = unicodedata.normalize('NFC', string)
    nfd = unicodedata.normalize('NFD', string)
    if nfc == nfd:
        return 'Plain'
    elif string == nfc:
        return 'NFC'
    elif string == nfd:
        return 'NFD'

    nfkc = unicodedata.normalize('NFKC', string)
    nfkd = unicodedata.normalize('NFKD', string)
    if string == nfkc:
        return 'NFKC'
    elif string == nfkd:
        return 'NFKD'
    else:
        return 'Unknown'

def GetRelativePath(plist):
    root = plist['$top']['root']
    indx = plist['$objects'][root]['RelativePath']
    return plist['$objects'][indx]

def SetRelativePath(plist, relativePath):
    root = plist['$top']['root']
    indx = plist['$objects'][root]['RelativePath']
    plist['$objects'][indx] = relativePath

import os
from termcolor import cprint

print('Checking for potential errors...')

db = ManifestDB()
db.open(r'Manifest_15.4.db')
items = db.files()
items = { f[0] : f for f in items}

print('- Item count:', len(items))

count = {}
for index, key in enumerate(items):
    fileID, domain, relativePath, flags, file = items[key]
    
    form = NormalizationForm(relativePath)
    count[form] = count.get(form, 0) + 1
    
    plistBlob = plistlib.loads(file)
    plistPath = GetRelativePath(plistBlob)
    
    if fileID != db.fileId(domain, relativePath):
        cprint(f'- Detected target item ID mismatch Origin: {fileID}', 'red')
        cprint(f'  * fileID: {fileID}=sha1({ f"{domain}-{relativePath}".encode() })', 'yellow')
        cprint(f'  - domain: {domain}')
        cprint(f'  - relativePath: {relativePath}')
        
    if plistPath != relativePath:
        cprint(f'- Detected target item path mismatch Origin: {fileID}', 'red')
        cprint(f'  - domain: {domain}')
        cprint(f'  - relativePath(UTF-8): {relativePath.encode()}')
        cprint(f'  * plistPath   (UTF-8): {plistPath.encode()}', 'yellow')

    pdir = os.path.dirname(relativePath) 
    if pdir:
        pid = db.fileId(domain, pdir)
        if pid not in items:
            cprint(f'- Detected parent directory of target item does not exist: {fileID}', 'red')
            cprint(f'  - domain: {domain}')
            cprint(f'  - relativePath: {relativePath}')
            cprint(f'  * ParentItem: {pid}=sha1({ f"{domain}-{pdir}".encode() })', 'yellow')
    
    if index % 5000 == 0:
        print(f'- {index:0<5} / {len(items)} ...')

print('Check completed, relativePath normalization form statistics:')
for form in count:
    print(f'- {form: <5}: {count[form]}')

输出:

Checking for potential errors...
- Item count: 100717
- 00000 / 100717 ...
- 50000 / 100717 ...
- 10000 / 100717 ...
- 15000 / 100717 ...
- 20000 / 100717 ...
- 25000 / 100717 ...
- Detected target item ID mismatch Origin: 84f50bb8bc96c5741ab156130ba4de8b4901c1c1
  * fileID: 84f50bb8bc96c5741ab156130ba4de8b4901c1c1=sha1(b'AppDomain-org.videolan.vlc-ios-Documents/Audio books/Starwars/4 New Republic Era (5 to 25 ABY)/36 The Hand of Thrawn Duology -\xd3\xb0ecter of the Past')
  - domain: AppDomain-org.videolan.vlc-ios
  - relativePath: Documents/Audio books/Starwars/4 New Republic Era (5 to 25 ABY)/36 The Hand of Thrawn Duology -Ӱecter of the Past
- Detected target item ID mismatch Origin: b8ae2d19ce590a182d23561dded5dc34f053e42e
  * fileID: b8ae2d19ce590a182d23561dded5dc34f053e42e=sha1(b'AppDomain-org.videolan.vlc-ios-Documents/Audio books/Starwars/4 New Republic Era (5 to 25 ABY)/22 The Jedi Academy Trilogy\xc3\xa8ampions of the Force')
  - domain: AppDomain-org.videolan.vlc-ios
  - relativePath: Documents/Audio books/Starwars/4 New Republic Era (5 to 25 ABY)/22 The Jedi Academy Trilogyèampions of the Force
- 30000 / 100717 ...
- 35000 / 100717 ...
- 40000 / 100717 ...
- 45000 / 100717 ...
- 50000 / 100717 ...
- 55000 / 100717 ...
- 60000 / 100717 ...
- 65000 / 100717 ...
- 70000 / 100717 ...
- 75000 / 100717 ...
- 80000 / 100717 ...
- 85000 / 100717 ...
- 90000 / 100717 ...
- 95000 / 100717 ...
- 100000 / 100717 ...
Check completed, relativePath normalization form statistics:
- Plain: 100701
- NFC  : 2
- NFD  : 14

经过以上脚本检查发现，其实上级项目是存在的，因为其中包含非 ASCII 字符，因此字符 è 是以 UTF-8编码来计算的 SHA1 值。但是在这里上级项目的 fileID 与 relativePath 对不上，应该是在备份时计算 fileID 与存储 relativePath 的编码规范发生了变化导致的问题。

可以看到出错的项目的文件路径使用的规范是 NFD （完全分解），而上级项目的 fileID 也是使用的 NFD 计算的，但是 relativePath 却是 NFC （完全组合），所以会导致文件路径不匹配。

那么确定了问题的原因，我们只需要将 fileID 不匹配的 relativePath 转换为 NFD 编码规范即可。

import shutil

shutil.copy2(r'Manifest_15.4.db', r'Manifest.db.fixed')

db = ManifestDB()
db.open(r'Manifest.db.fixed')

for fileID, domain, relativePath, flags, file in db.files():
    if fileID != db.fileId(domain, relativePath):
        cprint(f'- Process: {fileID}', 'yellow')

        relativePathNFD = unicodedata.normalize('NFD', relativePath)
        plistBlob = plistlib.loads(file)
        SetRelativePath(plistBlob, relativePathNFD)
        plistBlobNFD = plistlib.dumps(plistBlob, fmt=plistlib.FMT_BINARY)
        
        if fileID == db.fileId(domain, relativePathNFD):
            cprint(f'  - NFD: {relativePathNFD}', 'green')
            cprint(f'  - FIX: ', 'green')
            cprint(f'    * From(UTF-8): {relativePath.encode()}', 'green')
            cprint(f'    * To  (UTF-8): {relativePathNFD.encode()}', 'green')

            db.cursor.execute("UPDATE Files SET relativePath = ?, file = ? WHERE fileID = ?", (relativePathNFD, plistBlobNFD, fileID))

db.conn.commit()
db.close()
print('Finished!')

输出:

- Process: 84f50bb8bc96c5741ab156130ba4de8b4901c1c1
  - NFD: Documents/Audio books/Starwars/4 New Republic Era (5 to 25 ABY)/36 The Hand of Thrawn Duology -Ӱecter of the Past
  - FIX:
    * From(UTF-8): b'Documents/Audio books/Starwars/4 New Republic Era (5 to 25 ABY)/36 The Hand of Thrawn Duology -\xd3\xb0ecter of the Past'
    * To  (UTF-8): b'Documents/Audio books/Starwars/4 New Republic Era (5 to 25 ABY)/36 The Hand of Thrawn Duology -\xd0\xa3\xcc\x88ecter of the Past'
- Process: b8ae2d19ce590a182d23561dded5dc34f053e42e
  - NFD: Documents/Audio books/Starwars/4 New Republic Era (5 to 25 ABY)/22 The Jedi Academy Trilogyèampions of the Force
  - FIX:
    * From(UTF-8): b'Documents/Audio books/Starwars/4 New Republic Era (5 to 25 ABY)/22 The Jedi Academy Trilogy\xc3\xa8ampions of the Force'
    * To  (UTF-8): b'Documents/Audio books/Starwars/4 New Republic Era (5 to 25 ABY)/22 The Jedi Academy Trilogye\xcc\x80ampions of the Force'
Finished!

将 Manifest.db.fixed 文件提交给用户替换到原备份的 Manifest.db 文件后，成功解决了问题。

文件名规范引发的 iTunes 备份还原错误及解决方案

分析错误原因

修复问题

Read next

常见距离算法

libimobiledevice 笔记一

libimobiledevice 笔记二在 Cygwin 中编译

Comments ()

分析错误原因

修复问题

Read next

Comments ( )

Comments ()