Automating Over The Air Deployment for iPhone

Automating Over The Air Deployment for iPhone

Since the release of the iOS4 we are able to distribute iPhone applications “Over The Air” (i.e: directly downloading the application from the iPhone without using iTunes).
This greatly simplifies the deployment process especially for entreprises where iTunes is rarely a corporate tool. It also allows you to create your own enterprise App Store.
But until now it was a fully manual process : not anymore !
In this article we’ll see how to automate the deployment with a software factory, thus making the deployment more reliable and more productive.

The missing command line :

Over the air deployment is based on a new command in the build menu of Xcode called “build and archive” which packages the application with an embedded provisioning profile.
There have been many questions on how to integrate this feature in a continuous integration process.

The “xcodebuild” command is well known : it builds an Xcode project from the command line and generates an “.app” file.

You can use this command to build your application from a script ran by a software factory.
But the generated file cannot be distributed over the air since it misses the embedded provisioning profile.
We need to build the project into an “.ipa” file, containing the provisioning profile and signed with your developper identity.

To be able to find the command line, the trick was to watch the system console log while running a “build and archive” through Xcode. You will then see something similar to this :

Console log extract :
31/10/10 20:57:16 Xcode[16510] Running /usr/bin/xcrun with (
“-sdk”,
iphoneos,
PackageApplication,
“-v”,
“/Users/barbu/Library/MobileDevice/Archived Applications/FDA8B2FA-5AE1-43E7-BF9F-CD32FD258907.apparchive/TestTemplate.app”,
“-o”,
“/Users/barbu/Library/MobileDevice/Archived Applications/FDA8B2FA-5AE1-43E7-BF9F-CD32FD258907.apparchive/TestTemplate.ipa”,
“–sign”,
“iPhone Developer: M VINCENT DAUBRY (J9TS3TJRYX)”,
“–embed”,
“/Users/barbu/Library/MobileDevice/Provisioning Profiles/68D899AB-9FEB-4CBB-A080-1078FF2FABCF.mobileprovision”
)

Writing the script :

You can now run a script which will :

  • Build your application into an .app file
 xcodebuild -target "${PROJECT_NAME}" -sdk "${TARGET_SDK}" -configuration Release
  • Package it into an .ipa file
/usr/bin/xcrun -sdk iphoneos PackageApplication -v "${RELEASE_BUILDDIR}/${APPLICATION_NAME}.app" -o "${BUILD_HISTORY_DIR}/${APPLICATION_NAME}.ipa" --sign "${DEVELOPER_NAME}" --embed "${PROVISONING_PROFILE}”

Here is a sample build script.

EDIT: June 13th, 2012

Since this article has been published, the industrialization of iOS development has been improved. Specially with the release of the XCode plugin for Jenkins.

One year ago, OCTO launched the Appaloosa service, the private store service to help our teams, our customers and anybody who needs to go further with the Over-The-Air distribution of their private apps.

Appaloosa provides its own Jenkins plugin to automatically deploy the latest version of an application to the private store.

36 commentsfor “Automating Over The Air Deployment for iPhone”

  1. Looks like you have two typo `DEVELOPPER_NAME` and `PROVISONNING_PROFILE` in this line: “${BUILD_HISTORY_DIR}/${APPLICATION_NAME}.ipa” –sign “${DEVELOPPER_NAME}” –embed “${PROVISONNING_PROFILE}”

    Thanks for answering my StackOverflow question!

  2. I’m not clear on how this works. Do I save you build script as-is, or make changes? Where exactly do I invoke it from?

  3. Hi Scott,

    You have to change the environment variables (project name, IOS version, etc). They are at the top of the script.

    You can run this script from command line or from a continuous integration tool such as hudson.

    Vincent.

  4. Many thanks for this tip, one thing that tripped me up was trying to use relative paths int he script which cause it to succeed but leave my IPA in some weird /var/tmp location. So use absolute paths people!

  5. Nice post, I’ve taken an interest in automating the build and distribution process lately. I’ve got a Hudson rig running and packaging my apps for OTA distribution.

    I’m actually using a different technique though, which does not use the xcrun command. I use the following lines to package up the .ipa file, and coupled with an appropriately configured .plist, this has been sufficient for OTA distribution.

    mkdir Payload
    cp -rp build/Release-iphoneos/MyApp.app Payload/
    zip -r MyApp.ipa Payload
    rm -rf Payload

  6. Hello,

    I have similar problems here to run such commands and integrate them with Hudson. In fact, we can only build the app in Debug (and no codesign) config. We don’t reach the xcrun stage, but the xcodebuild command fails already. The error is that Xcode is unable to codesign the app. Anyone has an idea? It is not exactly the same problem as solved in this post, but closely related.

    Thanks!

  7. Hi,

    Hufson creates a hidden directory “.hudson” in your user directory. You will find the checked out xcode project there. Try opening it and build the project from xcode, you will see what’s going wrong.

    Hope this helps,
    Vincent

  8. How do we integrate this with hudson builds? I believe hudson generates the app files itself and creates the whole web front-end for you. How would you configure it to link to this ipa file?

  9. Hello,

    This article was specifically dealing with packaging and signing your application from the command line.
    It assumes that you have already a hudson job building to build an Xcode project (to do so, you have to create a custom job inside Xcode and run a shell script with this custom job.)
    Here is an article explaining this process :

    http://nachbaur.com/blog/how-to-automate-your-iphone-app-builds-with-hudson

    Hope this helps,
    Vincent.

  10. When I add this to my Hudson build, it makes the .ipa file just fine, but when I point to it from the browser, it just says “Download Failed. Safari cannot download this file”. But when I use XCode’s Build and Archive – (Distribute for Enterprise), I get to enter a URL, a title and all that good stuff, and it generates a plist file for me that I need to point to before it works. Any idea what I’m missing?

  11. Hi,

    You’re just one step from having this working : what you need to do now is to create the PList you’re talking about. You can either do it by hand (and use the script only for updating an existing application), or you can even do it within the script (but are you creating you new application every day, isn’t it going to take longer to write the script than to generate manually the next 10 project you’ll be working on.. ?)

    - If you want to create it by hand : just run the build and archive from Xcode as you describe. Put the ipa and Plist somewhere. And when you want to update your application just have the script generate a new ipa and replace the old one.

    - If you want to generate the Plist within the shell script : i suggest you look towards PList command line tools such as PlistBuddy :
    http://developer.apple.com/library/mac/#DOCUMENTATION/Darwin/Reference/ManPages/man8/PlistBuddy.8.html

    Hope this helps,
    Vincent

  12. Great! Thanks for all the help. Last question: how do I configure Hudson to convert the download link on the webpage to use instead of the usual

  13. Could you explain what download link you are talking about ?
    Actualy hudson is just running your script, so whatever you want to perform it wont be by configuring hudson. You should rather be looking at shell scripting tutorials.

  14. Hi Vincent,

    I enjoy your post as I am planning to integrate UIAutomation with Hudson and this proves helpful.
    Here are the details:
    – whenever a dev checks-in his code, hudson would take the lead and create builds of each iOS type.
    – then I would like to start the iOS simulator and subsequently run some Instruments templates for UIAutomation those.

    The problem is that I cannot get the simulator to start through command line. My command would be something like:

    xcrun -sdk iphonesimulator4.2 “” -v “/../debug-iphonesimulator”

    Can you tell me how to start the iOS simulator?

    Merci

  15. What about generating the OTA manifest file? Is that possible?

  16. Hi,

    The OTA manifest file can be generated through Xcode using “build and archive” -> share -> distribute for enterprise.

    There might be a way to do it from the command line, but as this file needs to be generated only once per project i don’t think there is any point automating this process.

    (if you need to update the content of the manifest, to change the version number for example, you can use the PlistBuddy command)

    Hope this helps,
    Vincent

  17. Hey Vincent

    I just got XCode 4 with 4.3. Now after a couple of mods, I was able to run the script. Now OTA will work on iPad with 4.2.1 but for 4.3 devices, the OTA ipa file installs but for some reason disappears, right after the install completes. However if I restart my iPad the app magically appears. Do you know whats going on? Also how in the world do I create ipa files using XCode 4. It creates some xarch…file. Its so hard for me to find info on this..
    thanks
    Pronob

  18. Hi,

    I have no idea why you have this problem, maybe you should restore your iPad with iTunes.

    Hope this helps,
    Vincent

  19. Hi Pronob,

    Please can you list the mods you made to get the approach to work with XCode 4?

    I get an error after codesign is complete:
    “error: Cannot read format tag from /var/folders/QR/QRgbPnxnFpqs9od6W9sNjL5P7qw/-Tmp-/siDo6YFpA7/entitlements_rawWA7x3EiU’” The entitlements_rawWA7x3EiU file is empty.

    Thanks,

    Aki

  20. Merci! The one thing that gave me trouble was that paths (e.g. BUILD_HISTORY_DIR) need to be absolute.

  21. Has anyone got this to work for uploading to testflight?

    I’m getting through both steps with no indication of any errors. I can upload to testflight; but the app then doesn’t completely download.

    Using Xcode 4.0.1, which apparently changes the Entitlements file a bit.

  22. Aki, did you fix your “error: Cannot read format tag from ….” error? I get exactly the same, but with XCode 3.2.4, so not just a problem with XCode 4 I guess.

  23. I am having trouble with the xcrun part and trying to embed the profile — seems to be an issue with the appname having a space… I have tried \ escaping etc etc — everything I can think of..
    APPNAME=”Dr. Csillag”

    I get
    error: Unable to copy ‘/Users/jvanvark/Library/MobileDevice/Provisioning\ Profiles/09314421-9C43-4207-BFB0-B263013361FD.mobileprovision’ to ‘/var/folders/zY/zYMtZNeGGNacUATBxDls7U+++TI/-Tmp-/DalULAfDVc/Payload/Dr. Csillag.app/embedded.mobileprovision’

    If I do this by hand and put in the \ for the space – it works fine… Any suggestions!?

  24. Nice work, this was useful and works for me.

  25. Hi Vencent,

    I have tried your script. Its working absolutely fine in every fine.Thanks for the nice tutorial and script. But lately I have added a dependency project in my main project and dependency project is basically a ZXing widget and ZXing library for QRCode scanning. Its compiling and working fine in the XCode. But when I try the script you have provided its failing.

    I am posting error

    cd /Users/Companyname/Development/ProjectName_1.0.0/iphone
    setenv LANG en_US.US-ASCII
    setenv PATH “/Developer/Platforms/iPhoneOS.platform/Developer/usr/bin:/Developer/usr/bin:/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/bin:/usr/X11/bin”
    /Developer/Platforms/iPhoneOS.platform/Developer/usr/bin/llvm-gcc-4.2 -x objective-c++ -arch armv7 -fmessage-length=0 -pipe -Wno-trigraphs -fpascal-strings -Os -Wreturn-type -Wunused-variable -DPHONEGAP_FRAMEWORK=YES -isysroot /Developer/Platforms/iPhoneOS.platform/Developer/SDKs/iPhoneOS4.3.sdk -gdwarf-2 -fvisibility=hidden -fvisibility-inlines-hidden -mthumb -miphoneos-version-min=3.1 -I/Users/Companyname/Development/ProjectName_1.0.0/iphone/build/ProjectName.build/Release-iphoneos/ProjectName.build/ProjectName.hmap -I/Users/Companyname/Development/ProjectName_1.0.0/iphone/build/Release-iphoneos/include -I/Users/Companyname/Development/ProjectName_1.0.0/iphone/build/ProjectName.build/Release-iphoneos/ProjectName.build/DerivedSources/armv7 -I/Users/Companyname/Development/ProjectName_1.0.0/iphone/build/ProjectName.build/Release-iphoneos/ProjectName.build/DerivedSources -F/Users/Companyname/Development/ProjectName_1.0.0/iphone/build/Release-iphoneos -F/Users/Shared/PhoneGap/Frameworks -include /var/folders/mZ/mZhhkYHAEAmvIdK006LaM++++TI/-Caches-/com.apple.Xcode.501/SharedPrecompiledHeaders/ProjectName-Prefix-aicfsokcunwhegduttyycdnurxjq/ProjectName-Prefix.pch -c /Users/Companyname/Development/ProjectName_1.0.0/iphone/ProjectName/BarcodeScanner.mm -o /Users/Companyname/Development/ProjectName_1.0.0/iphone/build/ProjectName.build/Release-iphoneos/ProjectName.build/Objects-normal/armv7/BarcodeScanner.o
    In file included from /Users/Companyname/Development/ProjectName_1.0.0/iphone/ProjectName/BarcodeScanner.mm:8:
    /Users/Companyname/Development/ProjectName_1.0.0/iphone/ProjectName/BarcodeScanner.h:10:34: error: ZXingWidgetController.h: No such file or directory
    /Users/Companyname/Development/ProjectName_1.0.0/iphone/ProjectName/BarcodeScanner.h:11:25: error: QRCodeReader.h: No such file or directory
    In file included from /Users/Companyname/Development/ProjectName_1.0.0/iphone/ProjectName/BarcodeScanner.mm:8:
    /Users/Companyname/Development/ProjectName_1.0.0/iphone/ProjectName/BarcodeScanner.h:15: error: cannot find protocol declaration for ‘ZXingDelegate’
    /Users/Companyname/Development/ProjectName_1.0.0/iphone/ProjectName/BarcodeScanner.mm: In function ‘void -[BarcodeScanner scan:withDict:](BarcodeScanner*, objc_selector*, NSMutableArray*, NSMutableDictionary*)’:
    /Users/Companyname/Development/ProjectName_1.0.0/iphone/ProjectName/BarcodeScanner.mm:30: error: ‘ZXingWidgetController’ was not declared in this scope
    /Users/Companyname/Development/ProjectName_1.0.0/iphone/ProjectName/BarcodeScanner.mm:30: error: ‘widgetController’ was not declared in this scope
    /Users/Companyname/Development/ProjectName_1.0.0/iphone/ProjectName/BarcodeScanner.mm:31: error: ‘QRCodeReader’ was not declared in this scope
    /Users/Companyname/Development/ProjectName_1.0.0/iphone/ProjectName/BarcodeScanner.mm:31: error: ‘qrcodeReader’ was not declared in this scope
    /Users/Companyname/Development/ProjectName_1.0.0/iphone/ProjectName/BarcodeScanner.mm: At global scope:
    /Users/Companyname/Development/ProjectName_1.0.0/iphone/ProjectName/BarcodeScanner.mm:40: error: expected type-specifier before ‘ZXingWidgetController’
    /Users/Companyname/Development/ProjectName_1.0.0/iphone/ProjectName/BarcodeScanner.mm:40: error: expected `)’ before ‘ZXingWidgetController’
    /Users/Companyname/Development/ProjectName_1.0.0/iphone/ProjectName/BarcodeScanner.mm:40: internal compiler error: tree check: expected class ‘type’, have ‘exceptional’ (error_mark) in objc_push_parm, at objc/objc-act.c:18143
    Please submit a full bug report,
    with preprocessed source if appropriate.
    See for instructions.

    Any lead is highly appreciable.

    Thanks and Regards,
    Amit Chabra

  26. Hi Amit,

    For some reason the compiler cannot find the library you’ve included :

    “error: ZXingWidgetController.h: No such file or directory”

    “error: ‘QRCodeReader’ was not declared in this scope”

    Anyway this is a problem specific to the library you’re using, you should try to submit your problem on the ZXing help group.

    P.S: It seems that some people have similar errors on the ZXing group :

    http://groups.google.com/group/zxing/browse_thread/thread/bf1dc07949baef28

    Hope this helps,
    Vincent.

  27. Hi Vencent,

    How to add a program organizer?

  28. Hi Oleg, what do you mean by “program organizer”.

    I dont see what it has to do with the subject of this article.

  29. Hi Vencent, sorry for my English. I mean an Organizer in XCode. When I use XCode’s Build and Archive – the app then enters in the organizer

  30. Hi Vincent,

    I’m trying to apply the technique you used to figure out what parameters PackageApplication accepts. You mentioned you were observing the system logs while performing the “build and archive”. Which logs were you looking at? I opened up “Console” app in utilities but I wasn’t able to observe any Xcode internal logs.

    Thanks!
    Tang

  31. Hi Tang,

    PackageApplication is a perl script so you can look at it in a regular text editor. It’s pretty readable and will show you all the options and steps. The script resides in /Developer/Platforms/iphoneos.Platforms/Developer/usr/bin/PackageApplication

    I’m writing this from memory so the “iphoneos.Platforms” maye be wrong but it should be obvious what the name is if you look in the directory. Actually, other parts of the path may wrong, knowing my memory. Just do a

    find /Developer -name PackageApplication

    and it will show you. :)

    Julian

  32. hi,

    does anyone know where to find the system console log?

    Jacky

  33. Hi,

    You can access the system log via the console app. Try to type “console” in spotlight and you should find it.

    Hope this helps,
    Vincent

  34. hi Vincent,

    Many thanks for your replay.

    I use the command line “tail -f /var/log/system.log” to follow my console system log. But I haven’t find the system console log while running a “build and archive” through Xcode.

  35. I get this error:

    Codesigning ” with ‘iPhone Developer: ‘
    + /usr/bin/codesign –force –preserve-metadata=identifier,entitlements,resource-rules –sign iPhone Developer: –resource-rules=/var/folders/nf/bz6q3hqd7n7bm2dt8l_1l2fw0000gn/T/cBrNET5m6D/Payload/abc.app/ResourceRules.plist /var/folders/nf/bz6q3hqd7n7bm2dt8l_1l2fw0000gn/T/cBrNET5m6D/Payload/abc.app
    Program /usr/bin/codesign returned 1 : [/var/folders/nf/bz6q3hqd7n7bm2dt8l_1l2fw0000gn/T/cBrNET5m6D/Payload/abc.app: replacing existing signature
    /var/folders/nf/bz6q3hqd7n7bm2dt8l_1l2fw0000gn/T/cBrNET5m6D/Payload/abc.app: object file format unrecognized, invalid, or unsuitable
    ]
    error: /usr/bin/codesign –force –preserve-metadata=identifier,entitlements,resource-rules –sign iPhone Developer: ) –resource-rules=/var/folders/nf/bz6q3hqd7n7bm2dt8l_1l2fw0000gn/T/cBrNET5m6D/Payload/abc.app/ResourceRules.plist /var/folders/nf/bz6q3hqd7n7bm2dt8l_1l2fw0000gn/T/cBrNET5m6D/Payload/abc.app failed with error 1. Output: /var/folders/nf/bz6q3hqd7n7bm2dt8l_1l2fw0000gn/T/cBrNET5m6D/Payload/abc.app: replacing existing signature
    /var/folders/nf/bz6q3hqd7n7bm2dt8l_1l2fw0000gn/T/cBrNET5m6D/Payload/abc.app: object file format unrecognized, invalid, or unsuitable

    Please help me, i am getting screwed.

  36. I m new to mac and xcode Pls tell me how to run your script and what change need to be made and how?

    Thanks in advance

Leave a comment