Connecting Skylight Apps to GitHub: the Devil is in the Details

As I mentioned in an earlier blog post, we recently rolled out the ability for Skylight app owners to connect their Skylight app to a GitHub repo in order to take advantage of the various permissions associated with it. This means you can now give access to a particular Skylight app to all GitHub users with access to a particular repo.

TL;DR: give all your devs instant access to Skylight via GitHub!

The Technical Stuff

The most challenging pieces of this task were:

1. working with the GitHub API to get the data we wanted
2. all the edge cases that kept popping up

You Can't Always Get What You Want... From an API

I picked up this feature from my co-worker Rocky, who did the initial research around the GitHub API to see what we could and couldn't do with it. We'd been using Octokit to more easily access the API from our Rails app, and it's mostly super easy to use! The main issues we ran into were when we cross-referenced the Octokit docs with the GitHub API docs. There were times we thought we literally could not acquire certain data when it turned out we just needed to format the request slightly differently.

For example: it was not immediately obvious to us that we actually had to include nil as an argument when using the Octokit repositories method with no user provided - i.e Octokit.repositories(nil, access_token: some_token) .

The documentation states that "if user is not supplied, repositories for the current authenticated user are returned" so we assumed either we were doing something wrong, or the Octokit docs were wrong. Turned out we just needed to supply nil in lieu of a user, but it took some doing for me to figure that out.

We encountered a lot of issues like this while working with the GitHub API and Octokit. We'd read in the docs that we could access certain data, but it would take us a while to figure out how to actually do so. Overall, both Octokit and the GitHub API are very well documented, so that's not a knock on them - it was more the case that the descriptions and/or examples weren't explicit enough to make it obvious that we were supposed to be doing one thing, so we ended up doing the wrong thing.

What Did We Need the GitHub API For, Anyway?

For the Skylight app owner connecting the app to a repo, we needed to see what organizations they belonged to and what repos they had access to through that organization.

For the Skylight user connecting through GitHub, we needed to check all the repos (public and private) that they had access to (admin or otherwise) against all the Skylight apps that had a repo attached to them. If we saw a match, we granted access to the repo. Similarly, if a user's repo access had changed, we removed access as well.

A challenger appears...

Above all else, the most challenging part about building this feature was the edge cases. There were so many, they were hard to keep track of! It took me about 3-4 extra days beyond my projection to actually finish this task, because every time I'd manually test it out, I'd come upon either another edge case I hadn't thought of before or a case that I could have sworn I wrote a test for, but the test hadn't covered this particular case.

Allowing a GitHub-connected owner of a Skylight app to connect one of their repos to their Skylight app was simple enough, they just choose a repo and we attach that repo's ID to the app to check against later.

When another user then signs in with GitHub, we check all the repositories they have access to against all the Skylight apps that are connected to a repo, then grant them access to those apps if the repo ID matched.

Then I had to account for new users signing up with GitHub for the first time. Not a problem. Basically the same deal.

THEN I had to account for users connecting their existing Skylight account to GitHub through the "connect to GitHub" button on the account settings page. Also fine. We handle most of these GitHub authentication events with a single method, so it wasn't too difficult to make small additions like this.

I had all the bases covered for adding an app to a GitHub user! But there was more. A lot more.

What if a user disconnects their account from GitHub? What should happen to the apps they were connected to through GitHub? 🤔

What if a user's repo access is revoked? 🤔

What if their admin access to a repo changes? 🤔

What if the app owner decides to connect the app to a different repo? Or no repo at all? 🤔

What if a user is invited to an app and would otherwise have GitHub access? 🤔

What happens if their GitHub access to that app is revoked (either because their repo access changed, the app's connected repo changed, or the app's repo ID was removed altogether)? 🤔

If we sync a user's GitHub access on login, what if a user stays logged in indefinitely while their repo access changes? Or the app removes or changes the repo ID associated with it during that time? 🤔

To be honest, I did account for most of these situations originally, but it was frustrating to deploy, get excited, test a thing, find a hole, fix the hole & write a test, deploy, get excited, rinse, repeat a couple of times.

Baby's First Background Job

Okay so, as a still relatively new developer, I had somehow not yet ever written a background job for any reason. It just hadn't come up! Finally, with this particular addition, I got the chance.

People can basically stay signed in to Skylight for a long, long time. It's completely possible that someone might log in using GitHub while they have access to certain repos that they no longer have access to a few days later, so we wanted to make sure we were periodically syncing everyone's GitHub access behind the scenes.

I wrote a job that, once a day, iterated over every single user that was connected to GitHub and synced their repos. Or so I thought. Don't try this at home:

class GithubRepoAccessWorker < DirewolfWorker

  def perform(user)
    user.sync_user_repo_access
  end
end

if Rails.application.dw_env == 'production'
  # run at midnight / every 24 hours
  User.github_connected.all.each do |user|
    Sidekiq::Cron::Job.create(name: 'Github Repo Access Worker - every night', cron: '0 0 * * *', class: 'GithubRepoAccessWorker', args: user)
  end
end

After the initial deploy, logging into our Heroku console yielded this horrifying sight:

With some guidance from my co-worker Peter, I rewrote it a bit. One to sync an individual user:

class GithubRepoAccessWorker < DirewolfWorker
  sidekiq_options queue: :low, unique: :until_executed, retry: 5

  def perform(user_id)
    user = User.find(user_id)
    user.sync_github_apps if user
  end
end

...and one to queue up that job once daily, passing in each GitHub-connected user's ID:

class QueueGithubRepoAccessWorker < DirewolfWorker

  def perform
    User.github_connected.pluck(:id).each {|id| GithubRepoAccessWorker.perform_async(id) }
  end
end

if Rails.application.dw_env == 'production'
  # run once daily
  Sidekiq::Cron::Job.create(name: 'Github Repo Access Worker - once daily', cron: '0 0 * * *', class: 'QueueGithubRepoAccessWorker')
end

We have not yet had any issues with these workers, but we'll see if this changes after deploying this update to all Skylight users. So far, so good!

More to come!

There are so many cool things we want to do with GitHub! This is really just the next step in our ongoing progress towards further integration with all the helpful things the GitHub API (and now, the GitHub GraphQL API) can offer.

If you want to stay updated on our GitHub integration process and be the first to know about (and test!) these kinds of features in the future, be sure and subscribe to Skylight Insider emails - log in to (or sign up for) Skylight, visit your account settings page, and scroll all the way to the bottom. Click "Insider", and you're done!

👋 See you next time! 👋