perf: eliminate full rescans with local caches; add queue tracking, notifications, SyncView redesign
Cached indexes:
- NASManifestCache: actor-based disk cache for the NAS manifest (kisani_manifest_cache.json)
Avoids NAS network round-trip when cache < 5 min old. Updated after every manifest download.
- LocalPhotoIndex: actor-based disk cache of phone asset metadata (kisani_local_index.json)
Built once on first launch, then updated incrementally via PHPhotoLibraryChangeObserver.
O(1) countSafe via in-memory dict (vs PHFetchResult batch enumeration + CoreData calls).
Fast path reconcile:
When both caches are warm, reconcile() skips NAS connection + PHFetchResult enumeration
entirely and returns in microseconds. Falls through to full NAS path only when stale.
AutoBackupCoordinator.onActive():
Changed force: true → force: false so the 30-second debounce prevents expensive rescans
on rapid app foreground/background cycles. Dashboard appears instantly from cached snapshot.
BackupQueueItem + queue tracking:
BackupEngine publishes queueItems: [BackupQueueItem] with per-file status, progress,
speed, error details. Queue is persisted to disk and reloaded on next launch so SyncView
always has data to show even after restart.
NotificationService:
Centralised local notification sender replacing the inline UNMutableNotificationContent
blocks. Sends for: backup started, completed, failed, paused, NAS offline. Uses category
identifiers so same-type notifications replace each other.
SyncView redesign:
- Removed pill/chip around NAS IP — replaced with plain "Online/Offline/Checking" dot+text
- No live NAS directory listing (was expensive NAS connection on every tab open)
- Shows engine.queueItems grouped into Active / Failed / Completed sections
- Per-file rows with inline progress bar + bytes/speed for uploading items, error text for failed
- Idle state shows archive count from statusService snapshot
GalleryViewModel:
NAS tab and All tab now use NASManifestCache as fallback when statusService.lastManifest
is nil (offline launch). Gallery shows NAS items even when NAS is unreachable.
AppDelegate:
Bootstraps LocalPhotoIndex.loadOrBuild() in background on launch (non-blocking).
Co-Authored-By: Kutesir <kutesir@provoc.ug>
Co-Authored-By: Sentry <sentry@provoc.ug>
2026-05-17 23:28:29 +03:00
|
|
|
|
import Foundation
|
|
|
|
|
|
import Photos
|
|
|
|
|
|
import os.log
|
|
|
|
|
|
|
|
|
|
|
|
private let log = Logger(subsystem: "com.albert.nasbackup", category: "LocalPhotoIndex")
|
|
|
|
|
|
|
|
|
|
|
|
/// On-device index of phone asset metadata — eliminates full PHFetchResult
|
|
|
|
|
|
/// enumeration for status counts after the initial build.
|
|
|
|
|
|
///
|
|
|
|
|
|
/// Build cost: O(N) on first launch, then zero on every subsequent open.
|
|
|
|
|
|
/// Update cost: O(changed) via PHPhotoLibraryChangeObserver.
|
|
|
|
|
|
/// Memory: ~100 bytes × asset count (≈1 MB for 10 k photos).
|
|
|
|
|
|
actor LocalPhotoIndex {
|
|
|
|
|
|
static let shared = LocalPhotoIndex()
|
|
|
|
|
|
|
|
|
|
|
|
struct Record: Codable, Sendable {
|
|
|
|
|
|
let localIdentifier: String
|
|
|
|
|
|
var filename: String
|
|
|
|
|
|
var creationDate: Date?
|
|
|
|
|
|
var mediaType: Int // PHAssetMediaType.rawValue
|
|
|
|
|
|
var isScreenshot: Bool
|
|
|
|
|
|
var isRAW: Bool
|
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
|
|
private var records: [String: Record] = [:]
|
|
|
|
|
|
private let diskURL: URL
|
|
|
|
|
|
private var needsSave = false
|
|
|
|
|
|
|
|
|
|
|
|
private init() {
|
|
|
|
|
|
let caches = FileManager.default.urls(for: .cachesDirectory, in: .userDomainMask)[0]
|
|
|
|
|
|
diskURL = caches.appendingPathComponent("kisani_local_index.json")
|
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
|
|
// MARK: — Lifecycle
|
|
|
|
|
|
|
|
|
|
|
|
/// Load from disk if available; otherwise build from PHFetchResult in background.
|
|
|
|
|
|
/// Safe to call multiple times — subsequent calls are no-ops if records are loaded.
|
|
|
|
|
|
func loadOrBuild() async {
|
|
|
|
|
|
guard records.isEmpty else { return }
|
|
|
|
|
|
if let loaded = await Self.loadFromDisk(url: diskURL) {
|
|
|
|
|
|
records = loaded
|
|
|
|
|
|
log.info("LocalPhotoIndex: loaded \(self.records.count) records from disk")
|
|
|
|
|
|
return
|
|
|
|
|
|
}
|
|
|
|
|
|
log.info("LocalPhotoIndex: building from scratch (first launch)")
|
|
|
|
|
|
await buildFromLibrary()
|
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
|
|
// MARK: — Queries
|
|
|
|
|
|
|
|
|
|
|
|
var totalCount: Int { records.count }
|
|
|
|
|
|
|
2026-05-18 10:01:09 +03:00
|
|
|
|
func count(filter: BackupFilter) -> Int {
|
|
|
|
|
|
records.values.filter { passes($0, filter: filter) }.count
|
|
|
|
|
|
}
|
|
|
|
|
|
|
perf: eliminate full rescans with local caches; add queue tracking, notifications, SyncView redesign
Cached indexes:
- NASManifestCache: actor-based disk cache for the NAS manifest (kisani_manifest_cache.json)
Avoids NAS network round-trip when cache < 5 min old. Updated after every manifest download.
- LocalPhotoIndex: actor-based disk cache of phone asset metadata (kisani_local_index.json)
Built once on first launch, then updated incrementally via PHPhotoLibraryChangeObserver.
O(1) countSafe via in-memory dict (vs PHFetchResult batch enumeration + CoreData calls).
Fast path reconcile:
When both caches are warm, reconcile() skips NAS connection + PHFetchResult enumeration
entirely and returns in microseconds. Falls through to full NAS path only when stale.
AutoBackupCoordinator.onActive():
Changed force: true → force: false so the 30-second debounce prevents expensive rescans
on rapid app foreground/background cycles. Dashboard appears instantly from cached snapshot.
BackupQueueItem + queue tracking:
BackupEngine publishes queueItems: [BackupQueueItem] with per-file status, progress,
speed, error details. Queue is persisted to disk and reloaded on next launch so SyncView
always has data to show even after restart.
NotificationService:
Centralised local notification sender replacing the inline UNMutableNotificationContent
blocks. Sends for: backup started, completed, failed, paused, NAS offline. Uses category
identifiers so same-type notifications replace each other.
SyncView redesign:
- Removed pill/chip around NAS IP — replaced with plain "Online/Offline/Checking" dot+text
- No live NAS directory listing (was expensive NAS connection on every tab open)
- Shows engine.queueItems grouped into Active / Failed / Completed sections
- Per-file rows with inline progress bar + bytes/speed for uploading items, error text for failed
- Idle state shows archive count from statusService snapshot
GalleryViewModel:
NAS tab and All tab now use NASManifestCache as fallback when statusService.lastManifest
is nil (offline launch). Gallery shows NAS items even when NAS is unreachable.
AppDelegate:
Bootstraps LocalPhotoIndex.loadOrBuild() in background on launch (non-blocking).
Co-Authored-By: Kutesir <kutesir@provoc.ug>
Co-Authored-By: Sentry <sentry@provoc.ug>
2026-05-17 23:28:29 +03:00
|
|
|
|
func allRecords() -> [Record] { Array(records.values) }
|
|
|
|
|
|
|
|
|
|
|
|
func countSafe(against index: ManifestIndex, filter: BackupFilter) -> Int {
|
|
|
|
|
|
var safe = 0
|
|
|
|
|
|
for record in records.values {
|
|
|
|
|
|
guard passes(record, filter: filter) else { continue }
|
|
|
|
|
|
if index.matches(localIdentifier: record.localIdentifier) {
|
|
|
|
|
|
safe += 1
|
|
|
|
|
|
} else if index.isFilenameOnly && index.matches(filename: record.filename) {
|
2026-05-18 00:17:16 +03:00
|
|
|
|
// Pure bootstrap mode: every entry is filename-only, match by name.
|
|
|
|
|
|
safe += 1
|
|
|
|
|
|
} else if index.matchesUnclaimed(filename: record.filename) {
|
|
|
|
|
|
// Mixed mode: this specific entry was uploaded without a localIdentifier
|
|
|
|
|
|
// (interrupted session / pre-ID backup). Filename match prevents re-upload.
|
perf: eliminate full rescans with local caches; add queue tracking, notifications, SyncView redesign
Cached indexes:
- NASManifestCache: actor-based disk cache for the NAS manifest (kisani_manifest_cache.json)
Avoids NAS network round-trip when cache < 5 min old. Updated after every manifest download.
- LocalPhotoIndex: actor-based disk cache of phone asset metadata (kisani_local_index.json)
Built once on first launch, then updated incrementally via PHPhotoLibraryChangeObserver.
O(1) countSafe via in-memory dict (vs PHFetchResult batch enumeration + CoreData calls).
Fast path reconcile:
When both caches are warm, reconcile() skips NAS connection + PHFetchResult enumeration
entirely and returns in microseconds. Falls through to full NAS path only when stale.
AutoBackupCoordinator.onActive():
Changed force: true → force: false so the 30-second debounce prevents expensive rescans
on rapid app foreground/background cycles. Dashboard appears instantly from cached snapshot.
BackupQueueItem + queue tracking:
BackupEngine publishes queueItems: [BackupQueueItem] with per-file status, progress,
speed, error details. Queue is persisted to disk and reloaded on next launch so SyncView
always has data to show even after restart.
NotificationService:
Centralised local notification sender replacing the inline UNMutableNotificationContent
blocks. Sends for: backup started, completed, failed, paused, NAS offline. Uses category
identifiers so same-type notifications replace each other.
SyncView redesign:
- Removed pill/chip around NAS IP — replaced with plain "Online/Offline/Checking" dot+text
- No live NAS directory listing (was expensive NAS connection on every tab open)
- Shows engine.queueItems grouped into Active / Failed / Completed sections
- Per-file rows with inline progress bar + bytes/speed for uploading items, error text for failed
- Idle state shows archive count from statusService snapshot
GalleryViewModel:
NAS tab and All tab now use NASManifestCache as fallback when statusService.lastManifest
is nil (offline launch). Gallery shows NAS items even when NAS is unreachable.
AppDelegate:
Bootstraps LocalPhotoIndex.loadOrBuild() in background on launch (non-blocking).
Co-Authored-By: Kutesir <kutesir@provoc.ug>
Co-Authored-By: Sentry <sentry@provoc.ug>
2026-05-17 23:28:29 +03:00
|
|
|
|
safe += 1
|
|
|
|
|
|
}
|
|
|
|
|
|
}
|
|
|
|
|
|
return safe
|
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
|
|
/// Returns localIdentifiers for assets not yet in the manifest (pending upload).
|
|
|
|
|
|
func pendingIDs(against index: ManifestIndex, filter: BackupFilter) -> [String] {
|
|
|
|
|
|
records.values.compactMap { record in
|
|
|
|
|
|
guard passes(record, filter: filter) else { return nil }
|
|
|
|
|
|
let alreadyBacked =
|
|
|
|
|
|
index.matches(localIdentifier: record.localIdentifier) ||
|
2026-05-18 00:17:16 +03:00
|
|
|
|
(index.isFilenameOnly && index.matches(filename: record.filename)) ||
|
|
|
|
|
|
index.matchesUnclaimed(filename: record.filename)
|
perf: eliminate full rescans with local caches; add queue tracking, notifications, SyncView redesign
Cached indexes:
- NASManifestCache: actor-based disk cache for the NAS manifest (kisani_manifest_cache.json)
Avoids NAS network round-trip when cache < 5 min old. Updated after every manifest download.
- LocalPhotoIndex: actor-based disk cache of phone asset metadata (kisani_local_index.json)
Built once on first launch, then updated incrementally via PHPhotoLibraryChangeObserver.
O(1) countSafe via in-memory dict (vs PHFetchResult batch enumeration + CoreData calls).
Fast path reconcile:
When both caches are warm, reconcile() skips NAS connection + PHFetchResult enumeration
entirely and returns in microseconds. Falls through to full NAS path only when stale.
AutoBackupCoordinator.onActive():
Changed force: true → force: false so the 30-second debounce prevents expensive rescans
on rapid app foreground/background cycles. Dashboard appears instantly from cached snapshot.
BackupQueueItem + queue tracking:
BackupEngine publishes queueItems: [BackupQueueItem] with per-file status, progress,
speed, error details. Queue is persisted to disk and reloaded on next launch so SyncView
always has data to show even after restart.
NotificationService:
Centralised local notification sender replacing the inline UNMutableNotificationContent
blocks. Sends for: backup started, completed, failed, paused, NAS offline. Uses category
identifiers so same-type notifications replace each other.
SyncView redesign:
- Removed pill/chip around NAS IP — replaced with plain "Online/Offline/Checking" dot+text
- No live NAS directory listing (was expensive NAS connection on every tab open)
- Shows engine.queueItems grouped into Active / Failed / Completed sections
- Per-file rows with inline progress bar + bytes/speed for uploading items, error text for failed
- Idle state shows archive count from statusService snapshot
GalleryViewModel:
NAS tab and All tab now use NASManifestCache as fallback when statusService.lastManifest
is nil (offline launch). Gallery shows NAS items even when NAS is unreachable.
AppDelegate:
Bootstraps LocalPhotoIndex.loadOrBuild() in background on launch (non-blocking).
Co-Authored-By: Kutesir <kutesir@provoc.ug>
Co-Authored-By: Sentry <sentry@provoc.ug>
2026-05-17 23:28:29 +03:00
|
|
|
|
return alreadyBacked ? nil : record.localIdentifier
|
|
|
|
|
|
}
|
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
|
|
// MARK: — Incremental updates (from PHPhotoLibraryChangeObserver)
|
|
|
|
|
|
|
|
|
|
|
|
func apply(inserted: [PHAsset], removed: [PHAsset], changed: [PHAsset]) {
|
|
|
|
|
|
for asset in removed { records.removeValue(forKey: asset.localIdentifier) }
|
|
|
|
|
|
for asset in inserted + changed {
|
|
|
|
|
|
records[asset.localIdentifier] = Self.makeRecord(asset)
|
|
|
|
|
|
}
|
|
|
|
|
|
needsSave = true
|
|
|
|
|
|
log.debug("LocalPhotoIndex: +\(inserted.count) -\(removed.count) ~\(changed.count)")
|
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
|
|
func flush() async {
|
|
|
|
|
|
guard needsSave else { return }
|
|
|
|
|
|
needsSave = false
|
|
|
|
|
|
let snapshot = records
|
|
|
|
|
|
let url = diskURL
|
|
|
|
|
|
await Task.detached(priority: .utility) {
|
|
|
|
|
|
guard let data = try? JSONEncoder.kisani.encode(snapshot) else { return }
|
|
|
|
|
|
try? data.write(to: url, options: .atomic)
|
|
|
|
|
|
}.value
|
|
|
|
|
|
log.debug("LocalPhotoIndex: flushed \(snapshot.count) records")
|
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
|
|
// MARK: — Private helpers
|
|
|
|
|
|
|
|
|
|
|
|
private func passes(_ record: Record, filter: BackupFilter) -> Bool {
|
|
|
|
|
|
let img = PHAssetMediaType.image.rawValue
|
|
|
|
|
|
let vid = PHAssetMediaType.video.rawValue
|
|
|
|
|
|
if !filter.includePhotos && record.mediaType == img { return false }
|
|
|
|
|
|
if !filter.includeVideos && record.mediaType == vid { return false }
|
|
|
|
|
|
if !filter.includeScreenshots && record.isScreenshot { return false }
|
|
|
|
|
|
if !filter.includeRAW && record.isRAW { return false }
|
|
|
|
|
|
return true
|
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
|
|
private func buildFromLibrary() async {
|
|
|
|
|
|
let opts = PHFetchOptions()
|
|
|
|
|
|
opts.includeAssetSourceTypes = [.typeUserLibrary, .typeCloudShared, .typeiTunesSynced]
|
|
|
|
|
|
let built: [String: Record] = await Task.detached(priority: .userInitiated) {
|
|
|
|
|
|
let result = PHAsset.fetchAssets(with: opts)
|
|
|
|
|
|
var map = [String: Record](minimumCapacity: result.count)
|
|
|
|
|
|
let batchSize = 200
|
|
|
|
|
|
var i = 0
|
|
|
|
|
|
while i < result.count {
|
|
|
|
|
|
autoreleasepool {
|
|
|
|
|
|
let end = min(i + batchSize, result.count)
|
|
|
|
|
|
for j in i..<end {
|
|
|
|
|
|
let asset = result.object(at: j)
|
|
|
|
|
|
let rec = Self.makeRecord(asset)
|
|
|
|
|
|
map[rec.localIdentifier] = rec
|
|
|
|
|
|
}
|
|
|
|
|
|
i = end
|
|
|
|
|
|
}
|
|
|
|
|
|
}
|
|
|
|
|
|
return map
|
|
|
|
|
|
}.value
|
|
|
|
|
|
records = built
|
|
|
|
|
|
needsSave = true
|
|
|
|
|
|
await flush()
|
|
|
|
|
|
log.info("LocalPhotoIndex: built \(self.records.count) records")
|
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
|
|
private nonisolated static func loadFromDisk(url: URL) async -> [String: Record]? {
|
|
|
|
|
|
await Task.detached(priority: .utility) {
|
|
|
|
|
|
guard let data = try? Data(contentsOf: url) else { return nil }
|
|
|
|
|
|
return try? JSONDecoder.kisani.decode([String: Record].self, from: data)
|
|
|
|
|
|
}.value
|
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
|
|
nonisolated static func makeRecord(_ asset: PHAsset) -> Record {
|
|
|
|
|
|
let resources = PHAssetResource.assetResources(for: asset)
|
|
|
|
|
|
let filename = resources.first?.originalFilename
|
|
|
|
|
|
?? "\(asset.localIdentifier.prefix(8)).jpg"
|
|
|
|
|
|
let isRAW = resources.contains { $0.type == .alternatePhoto }
|
|
|
|
|
|
let isScreenshot = asset.mediaSubtypes.contains(.photoScreenshot)
|
|
|
|
|
|
return Record(
|
|
|
|
|
|
localIdentifier: asset.localIdentifier,
|
|
|
|
|
|
filename: filename,
|
|
|
|
|
|
creationDate: asset.creationDate,
|
|
|
|
|
|
mediaType: asset.mediaType.rawValue,
|
|
|
|
|
|
isScreenshot: isScreenshot,
|
|
|
|
|
|
isRAW: isRAW
|
|
|
|
|
|
)
|
|
|
|
|
|
}
|
|
|
|
|
|
}
|