02. nov 2017 14:47 UTC
This is the story of a bug I found in
Daphne HTTP and Websocket terminating server, where I had to work around the bug for months while waiting for the fix to make it into a release.
Daphne is part of the
Channels project which is
Djangos cool websocket thing that recently got adopted as a part of the official
Django project. We've been using
Channels for a while on the the Schedule part of the BornHack website, which is how I found this
Daphne bug to begin with.
nginxproxy in front. This means that I make heavy use of the
X-Forwarded-*HTTP headers to see the real client IP in my requests. Basically the
nginxproxy receives the request, and does a new request to the backend
Djangoserver to get the response, and then passes that response back to the client. The proxy
nginxadds a couple of headers to the request called
X-Forwarded-Portand the backend
Djangoinstances can use those headers instead of the client ip (which points to the proxy).
The websocket requests handled by
Daphne were seen with the proxy IP as client IP, not respecting
X-Forwarded-For. This was weird because looking over the code it appeared to have support for those headers, and it turned out to be a bug related to Python2/3 support and encoding of a dict of headers. I fixed the bug back in April and did a pull request which was recently accepted into Daphne. But I needed a solution there and then, I couldn't sit around and wait for the next
What I needed was a way to get
pip to install the latest
Daphne 1.3.0 release (which is based on the
1.3.0 tag on
Github) with my patch on top. I could do this by hand of course, but that is not how I work. I need something I can put into
Ansible and deploy automatically and repeatedly without any fuzz. Fortunately it is easy to make
pip install a package from a Github branch. But first I need to Fork.
So the first thing I did was go to
Github and press
Fork on the Daphne repo. Then I cloned my new fork onto my laptop:
user@dev:~/devel$ git clone email@example.com:tykling/daphne.git Cloning into 'daphne'... remote: Counting objects: 1086, done. remote: Total 1086 (delta 0), reused 0 (delta 0), pack-reused 1086 Receiving objects: 100% (1086/1086), 239.07 KiB | 0 bytes/s, done. Resolving deltas: 100% (711/711), done.
I check that the
1.3.0 tag is present:
user@dev:~/devel/daphne$ git tag | grep 1.3.0 1.3.0
Cool. Then I make a new branch called
ws-x-forwarded-for-bugfix based on the upstream tag
1.3.0 to hold my bugfix:
user@dev:~/devel$ cd daphne/ user@dev:~/devel/daphne$ git checkout -b ws-x-forwarded-for-bugfix 1.3.0 Switched to a new branch 'ws-x-forwarded-for-bugfix' user@dev:~/devel/daphne$
git cherry-pick the latest commit from
master and add it to my branch:
user@dev:~/devel/daphne$ git cherry-pick master [ws-x-forwarded-for-bugfix fe7c7df] Make sure headers are always correctly encoded Date: Thu Oct 12 20:06:18 2017 +0200 1 file changed, 10 insertions(+), 5 deletions(-) user@dev:~/devel/daphne$
My bugfix just happened to be the latest commit on the
master branch of the upstream
Daphne repo, otherwise I would have to add some flags to the
git cherry-pick command.
Finally I push the new branch to my fork on Github:
user@dev:~/devel/daphne$ git push --set-upstream origin ws-x-forwarded-for-bugfix Counting objects: 4, done. Delta compression using up to 2 threads. Compressing objects: 100% (4/4), done. Writing objects: 100% (4/4), 654 bytes | 0 bytes/s, done. Total 4 (delta 3), reused 0 (delta 0) remote: Resolving deltas: 100% (3/3), completed with 3 local objects. To github.com:tykling/daphne.git * [new branch] ws-x-forwarded-for-bugfix -> ws-x-forwarded-for-bugfix Branch ws-x-forwarded-for-bugfix set up to track remote branch ws-x-forwarded-for-bugfix from origin. user@dev:~/devel/daphne$
All that remains is to make my deployments use my fork until there is a new release of
pip. This snippet from
requirements.txt shows how I disabled the regular
Daphne and made it use my fork from
#temp fix for https://github.com/django/daphne/pull/140 #daphne==1.3.0 git+https://github.com/tykling/daphne@ws-x-forwarded-for-bugfix
Thats all. It works and will keep things running until a new
Daphne release hits the streets. Until then!