Xcode Server Bot Email Triggers

Xcode server is typically run on a dedicated machine, e.g. a Mac Mini or cloud-based Mac server, so configuring the Xcode server to communicate the outcome of automations to the development team is critical to keeping forward progress on the development project.

In a previous post, I introduced Xcode Server bots and mentioned that an integration can fire an email trigger — for example to notify team members of successful integration completion or of exceptions encountered as the integration ran.

In that previous post I presented this diagram to illustrate an Xcode server automation that builds an app after a git branch is pushed to a remote repository on GitHub. Note the step in the bottom-right corner, which sends an email to the development team to communicate the outcome of the automation, or the overall status of the project.

Xcode Server automation example

What Do Email Triggers do for me?

Xcode server is typically run on a dedicated machine, e.g. a Mac Mini or cloud-based Mac server, so configuring the Xcode server to communicate the outcome of automations to the development team is critical to keeping forward progress on the development project.

An obvious choice to send status reports from Xcode server to development, QA and product teams is email, and Xcode server supports email reporting triggers out-of-the box.

Xcode Server provides two built-in email notification trigger types:

  1. New Issue Email. An email is sent when new issues are found. The intended recipient(s) for this type of trigger are the team members that introduced issues via source code contributions to the git repository.
  2. Periodic Email report. A summary of current project progress, intended to be sent to the broader team responsible for product development.

Configuring an automation email trigger

Configuring a trigger is really easy. While on the last tab of the integration configuration screen:

  1. Tap the plus (+) button to create a new trigger.
  2. Select either New Issue Email or Periodic Email Report.
  3. Select the types of issues that should trigger a new -mail each committer who introduced and issue.

In the case of a Periodic Email Report:

4. Select how frequently the summary report should be sent: after each integration, daily or weekly.

5. Since the trigger is a period report, the report can be sent to a broader audience, so you can add a list of email addresses/distribution lists in a way similar to sending an email from a standard mail client.

What do I get out of this?

After the bot is configured and saved (which sends it to the Xcode server for scheduling), Xcode Server will send emails according to the configuration steps made above.

Email will be delivered to your email box, and the delivered email will look similar to the following example of a compile error summary:

Example Xcode Server email report

Configuring your Xcode server to send email

The above discussion is focused on configuring bots with triggers that send email. But you do need to configure Xcode server so it has an authorized path to send the mail it’s configured to. I’ll discuss how to do this in the next post in this series: Configuring Xcode Server to send email.

Related to This Post

Configuring Xcode Server to Send Email

In my previous post on Xcode Server, I discussed the Xcode Server feature to send issue notification and summary email messages, and how to configure email triggers as part of integration configuration. In this post, I’ll discuss how to configure a MacOS Xcode Server machine to actually route email messages to the development team.

The Documented Way to Configure Xcode Server

As of this writing, the Xcode 10 documentation provides a single — and sparse — manual page on configuring Xcode Server to send email via SMTP:

Xcode Server Email Configuration Documentation (as of Xcode 10)

This seems very straightforward. You should add your outgoing SMTP server information and send from info — and Xcode server will send email. It should just work, right?

There’s only one problem with this procedure…for many people — including me — this doesn’t work.

Most likely this simple configuration can work in some environments, but Apple provides no information in the documentation regarding prerequisites and what type of SMTP environment this configuration is designed for.

No worries, though — next I’ll cover how to configure your Xcode server to work with any type of SMTP-based email infrastructure.

Note: if you know how to get the default Xcode configuration to work with authenticated, TLS-based email back-ends, please let me know in the comments!

Configuring Xcode Server to Send Email via Postfix

If you’ve installed Xcode 10 on macOS Mojave (and probably previous versions of both — though I’m only covering Xcode 10 here), your Xcode server hardware should already have a dormant installation of Postfix pre-installed. We’ll use that existing install of Postfix to get email delivery going with Xcode server.

Note: I use SMTP2GO as my SMTP provider for Xcode server, so the following instructions are specific to that service. This same procedure can work with other providers like Gmail, Office 365, and other services that provide SMTP with or without an encrypted SMTP connection. Adapt the settings below as required to match the requirements of your SMTP provider.

#1 Don’t configure Xcode’s email options

First, don’t configure Xcode’s email settings at all — just leave them blank. On your Xcode server, open Xcode Server configuration and ensure all fields are left blank, as in the documentation image above.

#2 Add your SMTP server credentials to the Postfix password file

Next configure your Xcode server Postfix installation with the username/password needed to authenticate with your SMTP server when sending e-mail.

Create or modify the /etc/postfix/sasl_passwd file using your favorite editor, for example:

$sudo nano /etc/postfix/sasl_passwd

Add a line that provides a valid username/password combination and corresponding server/port, with your own SMTP authentication credentials, similar to the following:

mail.smtp2go.com:2525 user@domain.com:password

#3 Run Postmap agains the password file

sudo postmap /etc/postfix/sasl_passwd

#4 Add Configurations to Postfix

Open the Postfix configuration file with your favorite text editor, for example nano:

$ sudo nano /etc/postfix/main.cf

Add correct configuration lines as required by your email provider. For smtptogo, I use the following:

relayhost = mail.smtp2go.com:2525
smtp_sasl_auth_enable=yes
smtp_sasl_password_maps=hash:/etc/postfix/sasl_passwd
smtp_use_tls=yes
smtp_tls_security_level=encrypt

Note that the relayhost in main.cf should match the url:port in the previous password file exactly.

It’s OK to add these configurations to the bottom of the main.cf file, but search the file and comment out any existing settings that would duplicate what you add.

#5 Start Postfix server on the Xcode Server machine

$ sudo postfix start

When Postfix is running, and your security configurations are correct — and the Xcode server mail settings are left blank — Xcode server will successfully send email when your email triggers are fired in Xcode integrations!

Related to This Post

CI/CD with Xcode Server 10

Xcode Server is a powerful and easy-to-use CI/CD solution that every Xcode 10 developer already installed — even though many of them don’t know it! In this article, I’ll overview the product architecture and discuss some of the plus and minus factors for using Xcode server vs. 3rd-party alternatives.

What does Xcode Server Do?

Like other CI/CD platforms, Xcode Server’s primary role is to automate the integration, analysis, unit testing, assembly and distribution of applications.

Typical use cases include:

  • Pulling an integration or distribution branch from a source code repo when commits are made (and/or on a nightly schedule).
  • Automatically run unit tests to verify that new code commits haven’t introduced regressions or performance problems.
  • Run static analysis of code to detect issues before application assembly.
  • Build QA or production application packages (archiving, in Apple speak).
  • Distribute completed archives to internal (ad-hoc) or external (Test Flight, Crashlytics, etc.) points.
  • Notify the Development, QA and Product teams of new build status, completions and exceptions.
  • Do all of the above continuously — perhaps several times per day — allowing developers to go back to work on their next tasks while these essential but repetitive tasks are completed by automated processes.
Xcode Server Build Process Overview

Many of the tasks Xcode server does are fully baked-in automation steps. Others are custom scripting tasks can be added to plug holes in the built-in capabilities — and add entirely new process steps limited only by a developer’s imagination.

Xcode Server History

First introduced by Apple with Xcode 5, Xcode Server is a first-party CI/CD solution — i.e. delivered and supported directly by Apple. When first introduced, Xcode server was one of many modules included in OS X Server (now known as macOS Server). In addition to CI/CD capabilities, OS X Server of that era included:

  • Email server
  • DNS Server
  • Git repository server
  • User profile management
  • And more…

Over time, Apple has pared back what is now branded macOS Server, removing many of the features that aren’t specific to macOS — and were probably under-utilized or not needed by customers. Today, macOS server still remains as a system administration layer over macOS for system administrator use, while Xcode server has been relocated into the Xcode.app product.

Xcode Server in Xcode

With Xcode 9 and Xcode 10, Xcode server is integrated with Xcode, rather than integrated with macOS server. This has several advantages:

  • Every installation of Xcode also installs Xcode Server and can be used directly
  • No need to license or install macOS Server on the remote integration server
  • Overall tighter integration with Xcode
  • A more familiar user experience for developers configuring Xcode Server.

With full Xcode integration, installing Xcode server really couldn’t be easier — just install Xcode, and Xcode server is installed as well. All that’s left is to enable it in preferences and select a user, on a new tab within the Xcode preferences screen.

Enabling Xcode Server in Xcode 10 Preferences

Running Locally or on an Integration Server

Xcode Server is installed along with Xcode 10, so does that mean your own development workstation can be your CI/CD server? The answer is — Yes! This is certainly possible, and may make sense for projects where the developer is working alone or with a very small team on a project — but would still like to take advantage of integration automations rather than running tasks manually.

Running on a Dedicated Integration Server

Probably more common is for a team of developers working on a product to use Xcode server as an automated integration point. This scenario doesn’t change typical developer workflow too much.

A lead developer or Devops staff would install and configure Xcode Server on a dedicated Mac, and then most developers would push code updates to remote git branches. Xcode server would then run its bot magic either on git push events, or on a scheduled event, e.g. nightly integration tests and builds.

Xcode server supports Subversion as well as git, though the latter is more commonly used today.

Developers contributing to a shared Xcode Server Installation

When used as part of an integration server deployment, it’s most common to deploy a dedicated Mac on a LAN — for example a headless Mac Mini dedicated to the task of fetching committed branches and running integration bots.

When deploying an integration server on a LAN isn’t a viable solution (e.g. remote teams), a Mac can be rented in the cloud, which is an economical way to deploy a headless Mac Mini in a professionally-managed data center. Popular Mac Mini hosting providers that can provide cloud-based hardware that even small teams can afford:

Mac Stadium

Mac In Cloud

XCLOUD

Currently there isn’t a platform-as-a-service (PAAS) offering for Xcode Server (a la Microsoft App Center or Circle CI). However, Apple’s recent acquisition and curtailment of the Buddy Build PaaS provider raises the obvious question: “is there an Xcode Server PAAS offering under development?” As with most new Apple product development, the answer is: “Nobody outside Apple knows!”

Xcode Server Alternatives

Even for iOS/macOS developers, Xcode Server isn’t the only option available. Popular open source tools or commercial services that can serve as viable Xcode Server alternatives include:

Fastlane

Jenkins

Microsoft App Center

Circle CI

Xcode Server Advantages compared withAlternatives

  • Xcode Server is arguably the easiest-to-use CI/CD solution for iOS or macOS application development. The software is already installed with Xcode, and gnarly issues like certificate management and build scripting is — for straightforward use cases — automatic and painless for the developer.
  • Apple supports and regression tests updates to Xcode server along with the Xcode product.
  • Except for the cost of dedicated server hardware (which is optional), Xcode server requires no additional up-front or ongoing operating costs for development teams.
  • Xcode Server can run unit and UI tests on physical iOS devices. Simply attach test iPhone/iPad devices to the Xcode Server, and add them to the test integration for the bot to run. Simple.

Xcode Server Disadvantages compared with Alternatives

  • Xcode Server is not cross-platform, and supports only Apple OS target applications.
  • The lack of a PaaS offering (at time of this writing) means to deploy Xcode Server requires you to provide hardware. This can take the form of a Mac mini (or other type of Mac) yourself, or renting a Mac from a cloud provider. However, this is also true of open source alternatives such as Fastlane/Jenkins — which are on-premises software too. It’s also true that the cost of a Mac may not exceed the cost of commercial PaaS offerings such as Circle CI or Microsoft App Center in the long-run.
  • Out-of-the-box, Xcode Server lacks some features found in other solutions. For example, Xcode Server (at time of writing) doesn’t include a built-in integration step for external deployment (e.g. Test Flight) — so deploying a finished archive to Test Flight is possible — but requires a custom post-integration script.

Is Xcode Server for you?

As always, the answer is: “maybe”. If the project you want to automate integration/testing/deployment for is targeting iOS or macOS, there’s really no reason not to try Xcode Server. It’s included with Xcode 10, and is a snap to setup and use.

Particularly if you’re new to CI/CD, you really can’t go wrong here — in the worst case you’ll get some experience with CI, probably cut down on some manual test/build work and better understand what features you need in a long-term solution if Xcode Server doesn’t fully meet your needs.

On the other hand, if you’re part of a fully-integrated cross-platform (e.g. iOS+Android) team that wants a unified solution for all development targets, Xcode Server might not be for you. While other solutions probably have a higher up-front learning curve and require more scripting to get going compared with Xcode Server, there are open source and commercial platforms that can provide cross-platform solutions where a unified CI/CD/Devops infrastructure is essential.

Related to This Post

Checking out AWS CodeCommit

Recently the git cloud server landscape became very interesting with the announcement by GitHub that the new pricing would be much more friendly toward users with many private repositories.

Recently the git cloud server landscape became very interesting with the announcement by GitHub that the new pricing would be much more friendly toward users with many private repositories.

Probably one reason for this is that the alternatives out there for low cost git hosting keep expanding, and GitHub must have seen the writing on the wall that it needed to get less stingy about private repositories.

But while GitHub is the most well known cloud repo offering, it’s not the only game in town. BitBucket, GitLab, and Microsoft Visual Studio Online are among other highly respectable code repository hosting platforms I’ve used that offer compelling services at similar prices.

Last year Amazon introduced its own Git-based repo offering, called CodeCommit. I’ve been intending to try it out, and finally have gotten around to it. Based on my initial experience, I’m glad I did!

AWS CodeCommit

I suspect one of the reasons pushing GitHub in th direction of free private repos is the introduction of CodeCommit by AWS last year. CodeCommit offers a very economical solution for private repos based on its uber-dependable AWS public cloud hosting platform.

While AWS isn’t providing all that GitHub has on offer — yet — it’s still a viable alternative for the basics of git hosting, and like everything else in AWS will probably evolve to compete beyond core functions over time.

The rest of this post is just a quick test I decided to do to create an XCode git repo on AWS CodeCommit in a similar way that I would usually do with GitHub. Then a quick summary with some thoughts at the end.

Uploading SSH Keys & Setting up IAM

As with GitHub and other git hosting services, the first step is to have a user account, and upload your public SSH key to that account.

Since CodeCommit is on AWS, the user account you need is an IAM Account. I already had one in my AWS account, so all I needed to do was to create a new SSH key pair on my Mac, add the public key to my IAM account on AWS, and add my IAM user to the policies as outlined in the getting started guide.

I found this process more complex than the same configuration on GitHub or BitBucket. For those not already familiar with AWS policies and the general flow of AWS portal administration, this could be off-putting. While I don’t ever remember reading the detailed instructions on GitHub (and I survived just fine), you definitely can’t skip reading the AWS CodeCommit documentation (though it is brief and well illustrated, so not a big chore). On the other hand, AWS is offering quite a bit more granular control and security with its platform — this is a trade-off. For my part I’ll gladly take extra security control in exchange for the effort of learning how to configure it.

Updating the SSH config file

The local side of the SSH authentication is to update the file ~/.ssh/config to let git know which private key to use when connecting to the CodeCommit server. These entries are similar to what you would have done using GitHub or BitBucket. Same process — this is a git thing, not an AWS thing.

Follow the instructions in points 10 and 11 in the the getting started guide, which explains how to do this quite clearly: CodeCommit Getting Started Guide. Thanks again AWS for great documentation!

Creating a shell repo

I tend to create a repo on a cloud provider’s system, and then connect my local git repo to it as a remote. CodeCommit supports this just like other providers. It also supports creating a remote and pulling the local from there — just like the providers.

Overall if you’ve used GitHub, this part of the process will feel very familiar.

Creating the repo can be done from the web console, which I found very simple. Alternatively, it can be done from the AWS CLI, which I found quite convenient, and easier to illustrate in a blog post, so I’ll use CLI from here.

My next step was to enter the command to create a repo, executed from any folder on my Mac:

$ aws codecommit create-repository --repository-name Test1 --repository-description "Quick test repo"

CodeCommit responded as such.

"Quick test repo" { "repositoryMetadata": { "repositoryName": "Test1", "cloneUrlSsh": "ssh://git-codecommit.us-east-1.amazonaws.com/v1/repos/Test1", "lastModifiedDate": 1463989097.607, "repositoryDescription": "Quick test repo", "cloneUrlHttp": "https://git-codecommit.us-east-1.amazonaws.com/v1/repos/Test1", "creationDate": 1463989097.607, "repositoryId": "e889d5cf-7c80-498e-ba56-884b468eba97", "Arn": "arn:aws:codecommit:us-east-1:************:Test1", "accountId": "*************" } }

So far, so good. Note the cloneUrlSsh, which is used n subsequent steps.

Connecting the XCode Project

I created my original project with XCode, and ticked the box to create a local git repository — so my local git work is 99% done, and all I need to do is add CodeConnect as a remote repo to the local one XCode created for me. Again, I’ll do that at the command line, but this time executed from the XCode project folder:

$ git remote add origin ssh://git-codecommit.us-east-1.amazonaws.com/v1/repos/Test1

There’s no response to this command, but for the curious, the effect of it can be checked in the .git/config file underneath the project folder:

[core] bare = false filemode = true ignorecase = true precomposeunicode = true logallrefupdates = true [core] repositoryformatversion = 0 [remote "origin"] url = ssh://git-codecommit.us-east-1.amazonaws.com/v1/repos/Test1 fetch = +refs/heads/*:refs/remotes/origin/*

Pushing the content

With all the setup complete, the proof is in the ability to push the repo content into AWS. Just for good measure, I’ll add files, execute a first commit, and then a push.

$ git add . $ git commit -m "First commit" $ git push origin master

Adding the ssh key passphrase to keychain

I did encounter one glitch — when trying to use git to push to the repo to the remote, I encountered a public key error, and my ssh key passphrase was failing to add to keychain via the interactive popup dialog box. To overcome this, I manually added the passphrase to keychain.

/usr/bin/ssh-add -k ~/.ssh/rob_kerr_aws_ssh

After the manual key add, the push worked just fine, and lightning fast:

Counting objects: 25, done. Delta compression using up to 4 threads. Compressing objects: 100% (21/21), done. Writing objects: 100% (25/25), 11.74 KiB | 0 bytes/s, done. Total 25 (delta 1), reused 0 (delta 0) remote: To ssh://git-codecommit.us-east-1.amazonaws.com/v1/repos/Test1 * [new branch] master -> master

Confirming the files made it to the remote repo

At this point all seemed to be OK, and the experience very much like using GitHub, BitBucket, or even a repo on OSX Server.

Just to check, I took a peek at the repo from the AWS portal — and found my files in the remote repo as expeted:

Generally I like AWS CodeCommit. Granted this entire try-out took just a few minutes, and I haven’t evaluated the full breadth of features nor used CodeCommit in a real project workflow as yet. But, for what it does, it looks pretty great, and the [CodeCommit pricing](CodeCommit pricing) for my use case would effectively be free — even less than the $7/month GitHub now charges for unlimited private repos on individual accounts.

Will I be switching to CodeCommit? Possibly. While GitHub provides more features and a more mature portal experience, at first glance CodeCommit seems to have the basic block and tackle hosted git covered, and the free tier limits seem to give me plenty of headroom. I’m happy to save some coin without losing many features I use. And since I use AWS already, I wouldn’t mind having one less cloud services account to keep track of.