Sync Realm Database with CloudKit

Overview

IceCream

CI Status Version Carthage compatible License Platform

contributions welcome

IceCream helps you sync Realm Database with CloudKit.

It works like magic!

Features

  • Realm Database

    • Off-line First
    • Thread Safety
    • Reactive Programming
    • Optimized for mobile apps
    • Easy when migrating
  • Apple CloudKit

    • Automatical Authentication
    • Silent Push
    • Free with limits(Private database consumes your user's iCloud quota)
  • Delta update

  • Reachability(Support Long-lived Operation)

  • Powerful Error Handling

  • Sync Automatically

  • Multiple object models support

  • Public/Private Database support

  • Large Data Syncing

  • Manually Synchronization is also supported

  • Relationship(To-One/To-Many) support

  • Available on every Apple platform(iOS/macOS/tvOS/watchOS)

  • Support Realm Lists of Natural Types

  • Complete Documentation

Prerequisite

  1. Be sure to have enrolled in Apple Developer Program
  2. Turn on your iCloud in Capabilities and choose CloudKit
  3. Turn on Background Modes and check Background fetch and Remote notification

Usage

Basics

  1. Prepare your Realm Objects (e.g. Dog, Cat...):
class Dog: Object {
    @objc dynamic var id = NSUUID().uuidString
    @objc dynamic var name = ""
    @objc dynamic var age = 0
    @objc dynamic var isDeleted = false

    static let AVATAR_KEY = "avatar"
    @objc dynamic var avatar: CreamAsset?

    @objc dynamic var owner: Person? // to-one relationships must be optional

    override class func primaryKey() -> String? {
        return "id"
    }
}
  1. Do stuff like this:
extension Dog: CKRecordConvertible & CKRecordRecoverable {
    // Leave it blank is all
}

Is that easy? Protocol Extensions do this trick.

  1. Start the Engine!
var syncEngine: SyncEngine?
func application(_ application: UIApplication, didFinishLaunchingWithOptions launchOptions: [UIApplicationLaunchOptionsKey: Any]?) -> Bool {
    ...
    syncEngine = SyncEngine(objects: [
            SyncObject(type: Dog.self),
            SyncObject(type: Cat.self),
            SyncObject(type: Person.self)
        ])
    application.registerForRemoteNotifications()
    ...
}
  1. Listen for remote notifications

The sample code in AppDelegate will be a good reference.

That's all you need to do! Every time you write to Realm, the SyncEngine will get notified and handle sync stuff!

For more details, clone the project to see the source code.

Object Deletions

Yep, we highly recommend you use Soft Deletions. That's why we add an isDeleted property to CKRecordConvertible protocol.

When you want to delete an object, you just need to set its isDeleted property to true and the rest of the things are already taken care of.

You also don't need to worry about cleaning-up things. It has also been considered.

How about syncing asset?

Luckily, we have a perfect solution for syncing asset. Absolutely, you could also store your image or kind of resource stuff as Data type and everything works fine. But Realm has a 16MB limit of data property. And CloudKit encourages us to use CKAsset in places where the data you want to assign to a field is more than a few kilobytes in size. So taking the consideration of the above two, we recommend you to use CreamAsset property to hold data. CreamAsset will store local data on the file system and just save file paths in the Realm, all automatically. And we'll wrap things up to upload to CloudKit as CKAsset.

An example project is provided to see the detailed usage.

Relationships

IceCream has officially supported Realm relationship(both one-to-one and one-to-many) since version 2.0.

Especially, for the support of to-many relationship, you have to pass the element type of the List to the SyncObject init method parameters. For example:

syncEngine = SyncEngine(objects: [
            SyncObject(type: Dog.self),
            SyncObject(type: Cat.self),
            SyncObject(type: Person.self, uListElementType: Cat.self) // if Person model has a List<Cat> property
        ])

Requirements

  • iOS 10.0+ / macOS 10.12+ / tvOS 10.0+ / watchOS 3.0+
  • Swift 5

Debug Suggestions

It's true that debugging CloudKit is hard and tedious. But I have some tips for you guys when facing puzzles:

  • You should know how Realm and CloudKit works.
  • Using GUI tools, like Realm Browser and CloudKit Dashboard.
  • When you are lost and don't remember where you are, I suggest starting all over again. In CloudKit Dashboard, "Reset..." button is provided. You can also clear local database by re-install apps.
  • By default, IceCream only prints some logs to your console in DEBUG mode. However, you could turn it off by adding IceCream.shared.enableLogging = false if it bothers you.
  • Keep calm and carry on!

Warning: If you're going to launch your app onto App Store, don't forget to deploy your environment settings to production. You can do it easily in the CloudKit Dashboard. Write & Read permissions are also need to be considered.

One More Tip

How to debug CloudKit in production mode? See this post.

Example

To run the example project, clone the repo, then open the Example/IceCream_Example.xcodeproj.

Installation Guide

Using Swift Package Manager, Carthage or CocoaPods.

Swift Package Manager

From Xcode 11, you can use Swift Package Manager to add IceCream and its dependencies to your project.

Select File > Swift Packages > Add Package Dependency. Enter https://github.com/caiyue1993/IceCream.git in the "Choose Package Repository" dialog. In the next page, specify the version resolving rule as "Up to Next Major" with "2.0.2" as its earliest version. After Xcode checking out the source and resolving the version, you can choose the "IceCream" library and add it to your app target.

If you encounter any problem or have a question on adding the package to an Xcode project, I suggest reading the Adding Package Dependencies to Your App guide article from Apple.

Carthage

Carthage is a decentralized dependency manager for Cocoa applications.

To integrate IceCream into your Xcode project using Carthage, specify it in your Cartfile:

github "caiyue1993/IceCream"

Then, run the following command to build the frameworks:

carthage update

Normally, you'll get IceCream, Realm and RealmSwift frameworks. You need to set up your Xcode project manually to add these 3 frameworks.

On your application targets’ General settings tab, in the Linked Frameworks and Libraries section, drag and drop each framework to use from the Carthage/Build folder on disk.

On your application targets’ Build Phases settings tab, click the “+” icon and choose “New Run Script Phase”. Create a Run Script with the following content:

/usr/local/bin/carthage copy-frameworks

and add the paths to the frameworks you want to use under “Input Files”(taking iOS platform for example):

$(SRCROOT)/Carthage/Build/iOS/IceCream.framework
$(SRCROOT)/Carthage/Build/iOS/Realm.framework
$(SRCROOT)/Carthage/Build/iOS/RealmSwift.framework

For more information about how to use Carthage, please see its project page.

CocoaPods

IceCream is available through CocoaPods. To install it, simply add the following line to your Podfile:

pod 'IceCream'

If you want to build IceCream as a static framework, CocoaPods 1.4.0+ is required.

Make it better

This is the to-do list for the IceCream project. You can join us to become a contributor.

  • CloudKit Shared Database

See the CONTRIBUTING file for contributing guidelines.

Live Demo

My app Sprint (A lightweight task management app) is using IceCream. You can download it and try it on your multiple devices to see this magic.

Reference

Contributors

This project exists thanks to all the people who contribute:

Sponsorship

Open source is great, but it takes time and efforts to maintain. I'd be greatly appreciated and motivated if you could to support the maintenance of IceCream financially. You could sponsor this project through the below ways:

And thanks to all our backers on open collective:

License

IceCream is available under the MIT license. See the LICENSE file for more info.

Issues
  • Offline support inquiry

    Offline support inquiry

    CloudKit recommends to track local changes for offline work.

    anyone else? 
    opened by harryworld 23
  • Realm Lists are being destroyed by SyncEngine

    Realm Lists are being destroyed by SyncEngine

    IceCream seems like an awesome solution to Realm/iCloud! I've got sync somewhat working. However I'm having a fatal issue.

    In my situation I need Dogs to have the option to have more than one Owner. So in Realm:

    Owner: let dogList = List<Dog>()

    Dog: let owners = LinkingObjects(fromType: Owner.self, property: "dogList")

    Everything works fine in Realm. When I create a new Dog, I do this:

            realm.add(dog)
            if let owners = owners {
                for owner in owners {
                   owner.dogList.append(dog)
               }
            }
    

    Once I add the code below code however, all of my dogs get removed from their owners.

    syncEngine = SyncEngine(objects: [
            SyncObject<Owner>(),
            SyncObject<Dog>(),
            ])
    

    The dogs are still visible in Realm Studio (they still exist in Realm), but their relationship to their owner(s) has been destroyed. All of my owners show zero objects in their dogList, whereas they had nonzero objects before SyncEngine runs. Do you know why this is happening? Again this only happens when I start the SyncEngine. If I comment that out, all of my relationships are maintained in subsequent app launches.

    opened by jln19 20
  • Use one SyncEngine for multiple Objects

    Use one SyncEngine for multiple Objects

    Reference to Realm Object Server, it registers the types to be synced to server once at launch.

    good first issue 
    opened by harryworld 19
  • Hangs when there are too many changes

    Hangs when there are too many changes

    My app gives the user the option to delete all his data and also to restore data from backup files.

    When erasing all data, I used realm.deleteAll(). Now that I'm using IceCream, I iterate over all my objects and set isDeleted to true.

    Then, when restoring data, I go over a different Realm database and use realm.create() to add it to the current Realm.

    Now, using IceCream, if I open my app in two devices at the same time, erase all data and then add it back from a backup file, the app will receive the cloudKitDataDidChangeRemotely hundreds of times and it'll hang forever, making it unusable and requiring me to force quit.

    In these cases I think it would be better to group multiple changes together and sync them on a background thread.

    good first issue 
    opened by dbmrq 18
  • Public database working

    Public database working

    This version appears to be working...

    Tell SyncEngine to use public database:

    let syncEngine = SyncEngine<MyObject>(usePublicDatabase: true)

    Mark objects to be stored in public database:

    extension MyObject: StoredInPublicDatabase {}

    opened by matadan 15
  • Implement background synchronization(closed by mistake)

    Implement background synchronization(closed by mistake)

    This resolves issue #88.

    I am unable to test this particular branch at the moment, as my personal fork of IceCream includes a few other changes (in order to get IceCream to work with App Groups and to fix some other sync issues. Both of which I will try to extract into PRs in the near future). I tried to extract only the background synchronization implementation. Background synchronization works wonderfully in my own project, so if I've extracted it correctly, it should work here as well.

    opened by kitlangton 14
  • Add CKAsset support

    Add CKAsset support

    This PR is for supporting CKAsset. Including a test demo.

    opened by MrFuFuFu 13
  • New ErrorHandler class

    New ErrorHandler class

    Implemented a new class called "CKErrorHandler" to switch on the generated error code and handle any and all potential CKError.codes should they arise.

    I modified SyncEngine to use the CKErrorHandler class where appropriate.

    enhancement 
    opened by randycarney 12
  • Crash  with longLivedOperation iOS 15

    Crash with longLivedOperation iOS 15

    Good afternoon, with the arrival of iOS 15 beta 4, there was a problem during synchronization and it seems that it will not go away. When SyncEngine is initialized, the method is called resumeLongLivedOperationIfPossible, which causes the application to crash. The crane happens when trying to add longLivedOperation in container, reason:

    Terminating app due to uncaught exception 'CKException', reason: 'CKDatabaseOperations must be submitted to a CKDatabase

    I attach the library code and the place of departure

    final class PrivateDatabaseManager {
        
        let container: CKContainer
        let database: CKDatabase
        
        public init(container: CKContainer) {
            self.container = container
            self.database = container.privateCloudDatabase
        }
        
        func resumeLongLivedOperationIfPossible() {
            container.fetchAllLongLivedOperationIDs { [weak self]( opeIDs, error) in
                guard let self = self, error == nil, let ids = opeIDs else { return }
                for id in ids {
                    self.container.fetchLongLivedOperation(withID: id, completionHandler: { [weak self](ope, error) in
                        guard let self = self, error == nil else { return }
                        if let modifyOp = ope as? CKModifyRecordsOperation {
                            modifyOp.modifyRecordsCompletionBlock = { (_,_,_) in
                                print("Resume modify records success!")
                            }
                            self.container.add(modifyOp) // CRASH HERE
                        }
                    })
                }
            }
        }
    }
    
    opened by alxrguz 11
  • Zone not found, first sync fail

    Zone not found, first sync fail

    Hey guys, this library is awesome thank you for creating it.
    I've noticed that when launching the SyncEngine the first time it syncs, I get the following errors:
    "ErrorHandler.Fail: Zone Not Found: the specified record zone does not exist on the server.CKError.Code: 26".

    I checked the code and I noticed that in the init, the createCustomZones function is called after syncRecordsToCloudKit, so I believe this is why I get the errors.

    The flow I use in the app is the following:
    iCloud is disabled by default, the user can add objects to realm and use the app.
    In Settings I've a toggle to turn iCloud on, when that happens the SyncEngine gets initialized.

    At the moment as a workaround I just call syncRecordsToCloudKit after the init and that solves the problem.

    Thank you.

    opened by timefrancesco 11
  • 一对多关系恢复问题

    一对多关系恢复问题

    我有一个类似于这样的一对多模型:

    class Person: Object {
        @objc dynamic var id = NSUUID().uuidString
        @objc dynamic var name = "Jim"
        @objc dynamic var isDeleted = false
        
        let dogs = LinkingObjects(fromType: Dog.self, property: "owner")
    }
    
    class Dog: Object {
        @objc dynamic var id = NSUUID().uuidString
        @objc dynamic var name = ""
        @objc dynamic var owner: Person?
    }
    

    当我拉取全部数据到本地时,我应该通过设置 Dogowner 属性来恢复一对多的关系

    realm.dynamicObject(ofType: ownerType, forPrimaryKey: primaryKeyValue)
    

    但在一种情况下,如果在 Person 对象还没有写入到本地的时候,通过 dynamicObject 来获取 Person 对象是不存在的,就不能正确的恢复这个关系。

    我查看了关于 list 这里关于解决关系恢复的问题,在单个对象属性设置的时候也应该需要相应的关系恢复逻辑。希望考虑解决一下。

    opened by huanglins 0
  • Cannot find type 'ListBase' in scope

    Cannot find type 'ListBase' in scope

    Compile the project, the following code reports an error

    /// We may get List here /// The item cannot be casted as List /// It can be casted at a low-level type ListBase guard let list = item as? ListBase, list.count > 0 else { break } var referenceArray = CKRecord.Reference let wrappedArray = list._rlmArray

    opened by Mr-ZNN 6
  • Synced records mismatch when Large sync operation

    Synced records mismatch when Large sync operation

    Expected behavior

    I want my objects to get synced correctly and stay correct

    Actual behavior(optional)

    When i sync a large amount of records (3k+) then some of them are not synced correctly.

    I have an app where the User can schedule Events, these events have a list containing Services The Services are only a reference in each event. this is because of storage and other things.

    The Service has a price property. When this property is changed, then all the events containing the Service gets updated by Realm, and therefore also by IceCream

    When this update happens, then sometimes the price property remains unchanged, but the Event shows the right price. So some data got synced some not.

    When i make that big update, i call syncEnginge.pushAll() I want to sync changes made on an iPad to my iPhone.

    These are the models im using:

    Events

    class  Event: Object {
    
    	@objc  dynamic  var  id = NSUUID().uuidString
    	@objc  dynamic  var  isDeleted: Bool = false
    
    	@objc  dynamic  var  eventName: String = "Minta Monika"
    	@objc  dynamic  var  honnanIsmerem: String? = ""
    	@objc  dynamic  var  eventLocation: String = "BP"
    	@objc  dynamic  var  eventStartDate: Date = Date()
    	@objc  dynamic  var  eventEndDate: Date = Date()
    
    	var eventJelzes = RealmProperty<Int?>()
    
    	@objc  dynamic  var  ismeteltEvent = false
    	@objc  dynamic  var  ismetlesID: String? = nil
    	var ismetlesGyakorisaga = RealmProperty<Int?>()
    	@objc  dynamic  var  ismetlesTipus: String? = nil
    	@objc  dynamic  var  meddigVanImetles: Date? = nil
    
    	var eventService = List<ArlistaElem>()
    
    	@objc  dynamic  var  eventPrice: Double = 0
    
    	var usedKupon = LinkingObjects(fromType: Kupon.self, property: "usedInEventsList")
    
    	override  class  func  primaryKey() -> String? {
    		return "id"
    	}
    }
    

    Services

    class ArlistaElem: Object {
    
    	@objc  dynamic  var id = NSUUID().uuidString
    	@objc dynamic var isDeleted: Bool = false
    
    	@objc dynamic var listaNev = "ListaNev"
    	@objc dynamic var listaAr: Double = 500
    	@objc dynamic var listaKategoria = "ListaKetegoria"
    	@objc dynamic var listaKedvenc = false
    
    	@objc dynamic var isArchived: Bool = false
    
    	var parentCategory = LinkingObjects(fromType: ArlistaKategoria.self, property: "kezelesLista")
    
    	override class func primaryKey() -> String? {
    		return "id"
    	}
    }
    

    Can someone help me? Because i could not get it to work after a week! I would really appreciate some good advice. Thank you!

    opened by elliotcz97 4
  • Pull inside a SwiftUI widget: does it work for you?

    Pull inside a SwiftUI widget: does it work for you?

    My app has a couple of Today Widget extensions in which I’m successfully invoking mySyncEngine.pull() method, resulting in an awesome user experience, since any remote updates are immediately available for the user.

    I’m now working on a new version of such widgets using SwiftUI. I’ve tried a similar approach inside my provider’s getTimeline() and/or getSnapshot() methods, however it doesn’t seem to have any effect: no updates are ever pulled. The widget’s entitlements look the same as the old widgets. The debugger doesn’t show any error messages: but I’m not sure I should trust on that lack of output.

    The only good news so far is that, when “Background App Refresh” is enabled on the device, remote updates received via silent push notifications will indeed update my local database and the widget will also reflect such updates on its next snapshot. But relying on that setting alone is not good news for me, since many users disable it and it’s also not available on macOS (while my app has a Mac Catalyst version).

    Has anyone been able to pull this feat (pun intended)? 😅 Any tips or suggestions are much appreciated.

    good first issue 
    opened by kleber-maia 0
  • Add Int64 list support

    Add Int64 list support

    In my current usage, Int64 list proprieties is getting lost during sync. This PR enable native List support.

    opened by raullermen 0
  • Crash with previous records containing non ASCII characters

    Crash with previous records containing non ASCII characters

    I just commented about this here: https://github.com/caiyue1993/IceCream/pull/191#issuecomment-822107108

    I have an app on the App Store running with IceCream for a while now, and my primary keys already had non ASCII characters. Now I tried upgrading to IceCream's latest version and it crashes when trying to sync because primary keys can't contain ASCII characters anymore. However, the records are already on iCloud with those keys! IceCream will fetch the records from iCloud and then crash because they contain those characters. But I can't even change them before fetching them.

    opened by dbmrq 2
  • Compatibility

    Compatibility

    opened by umidsaidov 0
  • Will IceCream Support Realm 10?

    Will IceCream Support Realm 10?

    Realm 10 is released and new data types are added. Will IceCream support Realm 10?

    opened by owenzhao 4
  • * fixed an issue that filePath may not be the same as the actual url of the asset.

    * fixed an issue that filePath may not be the same as the actual url of the asset.

    This issue is talked here. https://github.com/caiyue1993/IceCream/issues/217

    opened by owenzhao 0
  • Releases(2.0.3)
    Owner
    Soledad
    no is temporary, yes is forever.
    Soledad
    Sync Realm Database with CloudKit

    IceCream helps you sync Realm Database with CloudKit. It works like magic! Features Realm Database Off-line First Thread Safety Reactive Programming O

    Soledad 1.6k Sep 18, 2021
    Unrealm is an extension on RealmCocoa, which enables Swift native types to be saved in Realm.

    Unrealm enables you to easily store Swift native Classes, Structs and Enums into Realm . Stop inheriting from Object! Go for Protocol-Oriented program

    Artur  Mkrtchyan 459 Sep 14, 2021
    A library that provides the ability to import/export Realm files from a variety of data container formats.

    Realm Converter Realm Converter is an open source software utility framework to make it easier to get data both in and out of Realm. It has been built

    Realm 204 Jul 28, 2021
    Mac Native Mongodb Client

    System Requirements Mac OS X (10.8.x, 10.9.x, 10.10.x), intel 64bit based. Download HERE Or you can compile it yourself using Xcode Build Just build i

    Jérôme Lebel 2.5k Sep 12, 2021
    A Cocoa / Objective-C wrapper around SQLite

    FMDB v2.7 This is an Objective-C wrapper around SQLite. The FMDB Mailing List: https://groups.google.com/group/fmdb Read the SQLite FAQ: https://www.s

    August 13.6k Sep 21, 2021
    WCDB is a cross-platform database framework developed by WeChat.

    WCDB 中文版本请参看这里 WCDB is an efficient, complete, easy-to-use mobile database framework used in the WeChat application. It's currently available on iOS,

    Tencent 9.2k Sep 16, 2021
    YapDB is a collection/key/value store with a plugin architecture. It's built atop sqlite, for Swift & objective-c developers.

    YapDatabase is a collection/key/value store and so much more. It's built atop sqlite, for Swift & Objective-C developers, targeting macOS, iOS, tvOS &

    Yap Studios 3.3k Sep 14, 2021
    Official home of the DB Browser for SQLite (DB4S) project. Previously known as "SQLite Database Browser" and "Database Browser for SQLite". Website at:

    DB Browser for SQLite What it is DB Browser for SQLite (DB4S) is a high quality, visual, open source tool to create, design, and edit database files c

    null 15.4k Sep 24, 2021
    CoreData/Realm sweet wrapper written in Swift

    What is SugarRecord? SugarRecord is a persistence wrapper designed to make working with persistence solutions like CoreData in a much easier way. Than

    Modo 2.1k Aug 18, 2021
    Free universal database tool and SQL client

    DBeaver Free multi-platform database tool for developers, SQL programmers, database administrators and analysts. Supports any database which has JDBC

    DBeaver 22.2k Sep 16, 2021
    Native MongoDB driver for Swift, written in Swift

    Installation | Tutorial | Basic usage | About BSON | Codable | Community | How to help A fast, pure swift MongoDB driver based on Swift NIO built for

    null 617 Sep 20, 2021
    Personal db information management system.

    The Information Management System This tool is so simple that most people don't understand how to use it. TL;DR This is a personal database which stor

    null 39 Sep 14, 2021
    The easiest way to get started with PostgreSQL on the Mac

    Postgres.app The easiest way to run PostgreSQL on your Mac Includes everything you need to get started with PostgreSQL Comes with a pretty GUI to star

    Postgres.app 6k Sep 23, 2021
    Native cross-platform MongoDB management tool

    About Robo 3T Robo 3T (formerly Robomongo *) is a shell-centric cross-platform MongoDB management tool. Unlike most other MongoDB admin UI tools, Robo

    Studio 3T 8.7k Sep 23, 2021
    MySQL/MariaDB database management for macOS

    Sequel Ace Sequel Ace is the "sequel" to longtime macOS tool Sequel Pro. Sequel Ace is a fast, easy-to-use Mac database management application for wor

    Sequel-Ace 3.6k Sep 18, 2021
    MySQL/MariaDB database management for macOS

    Sequel Pro Sequel Pro is a fast, easy-to-use Mac database management application for working with MySQL & MariaDB databases. You can find more details

    Sequel Pro 8.5k Sep 19, 2021
    A type-safe, Swift-language layer over SQLite3.

    SQLite.swift A type-safe, Swift-language layer over SQLite3. SQLite.swift provides compile-time confidence in SQL statement syntax and intent. Feature

    Stephen Celis 7.8k Sep 16, 2021
    💻 Medis is a beautiful, easy-to-use Mac database management application for Redis.

    Medis Medis is a beautiful, easy-to-use Redis management application built on the modern web with Electron, React, and Redux. It's powered by many awe

    Zihua Li 10.4k Sep 14, 2021
    :wrench: Cross-platform GUI management tool for Redis

    RDM Install & Run | Quick Start | Native Formatters | Development Guide | Known issues | Telegram Chat Open source cross-platform Desktop Manager for

    Igor Malinovskiy 19k Sep 24, 2021