[onionmx] notes from onionmx meeting (01/03/2017)

[George K.]

Hello all,

some of us joined an online meeting to discuss a few issues regarding
onionmx. We promise to make the next meeting public to the list so more people can join!

Here are the notes of the meeting (thanks to micah for writing most of these!)
===========================================================================================

Agenda:

* state of the onionmx
* Tor support for SRV queries on DNSPort ? Updates on this...
* plan for an RFC-like document that will describe why we use SRV record, what will SRV weight and priority mean for every implementation, etc
* answer open github issues.
* decide upon what stays in the github repository and what not.
* rethink documentation/installation instructions
* standardize features among implementations (postfix w/ python, postfix w/ go, exim4)
* discuss packaging
* discuss promotion of onionmx (maybe some talk(s) in conferences?)
* irc channel / list ?


state of the onionmx
--------------------

There was a meeting at congress (33c3).
kargig and ng met after congress and discussed some things
immerda fixed a few things as they were discovered
david called out to the list
we maybe should have organized this meeting on that list - broaden the range of participants a little bit
git: https://github.com/ehloonion/onionmx
mailing list: onionmx at lists.immerda.ch (https://lists.immerda.ch/mailman/listinfo/onionmx)

state of the python daemon
-------------------------------------
added daemon-mode so postfix can communicate with the daemon (tcp socket is opened and postfix can communicate through there), also added an interactive (test mode where you make a DNS query and see the result of the onion resolution and the result that postfix would get if it were running in daemon mode)
debug mode: running as a daemon and spitting to STDOUT the queries and the answers (so you can see the queries that postfix makes and the replies)
client mode: for testing purposes mainly. You need to have a daemon running, and you can connect a client as if you were using netcat for example
also added functionality to support static routes: meaning you can put one or multiple yaml files into the sources directory and if an entry is there, it will not leak any DNS resolutions out and instead use the ones from the yaml
files. There is support for one or multiple yaml files. There is a case where you have the same entry in multiple yaml files you will get a random result (due to the yaml structure)... not sure if we should care about this, or not
at some point there will be better documentation and a package. But at this point there is a separate branch where ioparaskev is testing, when that is finished, we will clear up the documentation (branch: socket support)

go daemon
---------------
ale: go daemon is pretty much the same as it was before, it works (although ale not using it in production yet). init scripts/systemd units have been added, someone asked for them.

exim
----
ng: the exim version is running and documented in the immerda wiki entry (linked from the github project). Implemented postdns/go router in exim. It has a few routers: 1. one to one mapping of email domain and onion service. 2. do a MX
lookup and if the mail server is within the static mapping, then we deliver it to the onion service noted 3. then do a SRV record lookup and if its there deliver through that. The DNS lookup does go over normal DNS, so there are a few
leaks here, but as soon as exim decides to route an email through an onion service, from then on it goes through tor. There were some problems with DNS lookups through tor: verified in the RFCs that DNS servers don't really need to
respond to a MX or a SRV record, they can just respond with SRVFAIL. Exim sees this as a temporary problem and then queues the mail and tries over and over. Some legacy university servers stayed in the queue because the DNS servers that
they had didn't respond to MX or SRV record lookups. This could affect the postdns/go router implementation. Fixed by telling exim to treat a temporary DNS failure as a perm failure so it doesn't queue up... Unfortunate solution but as
long as we don't have any secure/reliable DNS lookup then we don't have any other way of dealing with that except for the static lookup.

Domains that were failing phd.ceu.hu / sowi.hu-berlin.de 

Tor DNS lookups
-----------------------
we need a way to do these lookups not over the clearnet. One way to do this is using a local resolver that forwards queries over tor to another node. Another way is to use tor's DNSPort for SRV lookup, but it currently cannot do this.
This is because it uses libevent and this doesn't have SRV lookup support. Nick Matthewson (tor dev and libevent maintainer) is looking for someone to work on changing how libevent does DNS queries, and maybe support more query types.
Maybe if we did a patch it could be accepted by libevent, could get Nick to do this if we had a patch. Micah will go to the tor-dev meeting in AMS in end of March, can talk to people about it there. We need someone with the skills to
implement this, it would help anonymize the DNS calls. There are some patches already to libevent, that have not been accepted because Nick wants to overhaul the library.
example DNS-related PR that has not been merged to libevent: https://github.com/libevent/libevent/pull/6

TODO: micah will talk to dgoulet to see if he'd be interested in doing this

RFC-like document for SRV lookups
-------------------------------------------------
ng: we've talked about writing an RFC for SRV lookups: better solution than low/high priority MX lookups, so a better way to go. Maybe before we add additional lookups to libevent, it might be good to make sure we are aligned around this being the right format for a lookup
kargig: should continue to use SRV lookups because it will be harder to talk to people who develop SMTP servers and get them to change their code to handle .onion MX replies/services differently (eg. getting SOCKS support in postfix)
ng: thinks we should stick with the SRV, and it would be a good idea to start writing an RFC-style document and put it into the repository. Could talk to dkg about this

TODO: ng will start working on such a document and discuss with dkg (but a couple weeks out from now)

kargig: Before writing anything, need to decide a few technical things, such as SRV weights and priorities, eg what happens when there are two records for a domain.
ng: would go with a high-level strategy approach: SRV says a few things about weight, priority and port numbers and we need to discuss how we handle these, because at the moment our implementation is straight forward and we expect one
result and we use that. In the exim implementation it is sorted by priority (but not weight). Would propose to look more into detail of how SRV is suggesting to use these fields and then follow these suggestions and write the RFC so it
is compatible with all/most protocols are doing. It could be a DNS lookup returns two SRV records with different weights and we might try two and only the second one succeeds. Maybe go into more detail when we have a proposal...
And add to the existing open issue about why not a MX record, already closed the issue and pointed to the mailing list and said we are working on a more formal document on SRV records and how we envision them to be used.
People ok with that response to that issue?
kargig: ok with that answer


Answering open github issues
------------------------------------------
Some of the issues are open for a few months. People should feel open to replying to these and not be scared.

#6 tls verification: https://github.com/ehloonion/onionmx/issues/6 - the user suggests we verify the onion with a cert -- mistaken what a certificate of a MX server should look like. Maybe post a reply that says we will publish a document at some point, so wait for that, to give a sign of life. 
https://github.com/ehloonion/onionmx/issues/14 - add doc label and once the documentation is there, then close it pointing to that


decide upon what stays in the github repository and what not.
-----------------------------------------------------------------------------------
We have a lot of different things around, documentation, code, yaml files, etc.  no formal structure
ng would like to move the blog post to the git repository

Maybe put all the documentation into a documentation folder
should someone be able to setup everything by just cloning this repository? or maybe packaging...
Maybe it makes more sense to separate the code as it becomes more complex and everything else can stay in this repository?
Add more repositories for the different implementations
proposal: keep documentation and ruby script that converts the static map to postfix (and the example static map) and the rest into separate repositories
maybe just put things in subfolders? 
having separate repositories helps with having separate issues with the protocol, idea and implementations, on the other hand it scatters the issues

People generally prefer to go with separate repositories (to the existing ehloonion group), we leave the ruby script/yaml data in the original repository for now, but re-evaluate later if there are further implementations
micah will make the documentation folder changes and re-link things into different sections

rethink documentation/installation instructions
---------------------------------------------------------------
First we restructure and then work over the documentation.

Discussion about structuring the flow of the documentation: do we remove the static map to discourage people from making pull requests to add them?
exim: static lookup, MX to static lookup, then does SRV lookup
postfix: 
changing the onion in the static map could be quite painful for people, there is a change coming for new onion services hostnames
a new user should have an easy way to test when setting it up, before setting up their own DNS SRV record to accept incoming emails. 
advertise the SRV record as a default, and make the static map as a more advanced step using our current file as an example....
Keep the static map around until we finalize the SRV record method and then we phase that out

standardize features among implementations
--------------------------------------------------------------
Probably need to document things first that describes the different features and once we have that we can make sure the implementations can do the same thing.
Maybe all the implementations dont have to do all of the same thing, but people should have an idea of what is expected to accomplish

discuss packaging
--------------------------
ng: one of the things missing we would need to add a license to things, for exim could come up with some snippets that would be installed as a package
go could be packaged with some helper scripts fairly easily, and could be packaged
python stuff could be packaged easily as well, no extreme dependencies, maybe will try to do it sometime end of next week

discuss promotion of onionmx (maybe some talk(s) in conferences?)
-----------------------------------------------------------------------------------------------
micah: can discuss at the tor-dev meeting coming up
ng: We should not wait until we have a more formal definition, that way we can gather feedback and it will push us to refine things and then we dont wait for the perfect documentation
kargig: would be nice to get some feedback outside of our smaller group, before making big promotional steps (announcing it on twitter, etc.)
having more smaller group communication/discussions/feedback and refine things before making larger promotional steps
IFF: meskio knows some people organizing a gathering of service providers, so will see if people are interested in the topic there
meskio will poke kali about maybe discussion with PANORAMIX people could happen: panoramix-project.eu
there is a mixnetwork meeting on the 28th in AMS at Greenhost


irc channel / list
---------------------
#onionmx on indymedia

===========================================================================================


Please comment!
-- 
George K.