POSTS

Voting machine POC

This blog post describes a POC of a Voting Machine. It may look more like a Roadmap or a backlog for myDHT, yet it still is an example of a myDHT application.

The code is hosted on github.

The protocol

Description

A simple protocol, between each step should be added synchro period where we check distribution or synchro of each peers but it is not done at this point :

  • a vote description is initiated, with a subject, and possible answers. It is given an Id. It is issued for specific users (votant). At this point wether it is public vote description or not depends. If public, the commitment of a secret key distributed to votant should be added to this description (mitigate exterior false envelope but votant could still distribute the secret key or emit multiple envelope).

  • we enter envelope period : each peer with access to the vote description will generate a Envelope (key pair) and share it (not the private key obviously) anonymously. Anomously is very important here : there must be no easy way to relate an envelope with a votant. The key on the envelope are mostly from an initial design where we did not add a commitment of the vote in the envelope. Here we choose to add such a commitment in the envelope (crypto hash of vote value and a random nonce), and the key on envelope is useless unless we want to ratach other info than the vote to this envelope later.

  • At the end of envelope period (no new envelope expected), every peer search for all envelope, and check them.

  • We enter participation period : peer sign the envelopes and send their participation (this sign).

  • End of participation period, peer vote if all participant and envelope seems ok with him (number of envelope/number of participant, same envelope report in all participation…), if wrong he transmit the issue somehow and vote stop (need to reemit envelopes and possibly republished a vote description with new secret key).

  • Anonymous send of its vote reply (signed by its own envelope pkey and if using a comitment in envolope, add the commitment nonce to this vote).

  • End of vote periode. Every voter can open all replies and update their vote results (after valdating replies of course (no dup… two reply for an enveloppe invalidate the envelope…)).

  • Synchro of results between votant to reach consensus. If using a commitment this step is less useful as it is harder to send a vote afterwards to change/invalidate the vote (unsing signing from envelope we could issue different reply).

In short it is simply a protocol where votes are sent anonymously and where every users verify individually their votes presence to validate the process. The fact that we need confirmation from every votant make it somehow unpracticable.

Issues

  • Main issue with this protocol is the validation of envelope. Envelope being publicaly signed (hash), any enveloppe can be emit (by a cheating participant or from a non participant if votedesc is leaked).
  • Vote signed by envelope is a little overdoing it : the purpose is to illustrate some striple usage. Putting a hash of the vote plus a nonce (commitment) in the envelope is enough. Yet looking at how frequently participation synch is likely to fail not even a commitment is harmless (failure of rng for commitment without impact). Their is also a caveat here : envelope could be use to produce multiple vote so the initial commitment may look better except if we say that multiple vote from an enveloppe invalidate this vote (thus the need to synchronize the vote).

  • Anonymous send is probably one of the main weakness : in the poc we will use some form of tunneling. With some non interactive zero knowledge proof their might be something to do with some unique secret key distribution between peers and a bloom filter circuit for proof, but it would require that the resulting zkproof got some unicity property to be of some interest, I do not know such a proof.

  • The temporality (limit each steps) is fine if we ensure that every votant are able to access the network fairly. Blockchain based publication is an easy way to manage it but this POC is intended to stay out of chain (a votedesc context could be seen as small sidechain), but at some point it is good idea at least publish some end of period state on a existing chain to facilitate and prove access to info.

The POC

Why a POC (Proof Of Concept) ?

  • MyDHT is far from usable and too generic in many ways. So beta trying it is the best way to get some roadmap.
  • Vote protocol is nice for its simplicity (scaling is something else and the envelope synchronisation scenario is not practical due to the ease of DDosing it (a single malicious votant is enough)). It may also be reuse to experiment some different approach of running scenario where a votant cheat and we isolate this through subvoting (trying to not compromise to much the vote anonymousity).
  • Involves some anonymity so a nice test for ‘mydht-tunnel’ crate.
  • Use of striple, striple is an old lib that I write but except for its command line usage I never really use it.

Should anyone use it?

Of course not, it is a POC and I believe this post will show that it is far from usable regarding the current state of myDHT.

So the POC is intended to be a straight forward implementation of the protocol. The goals being to demonstrate a Mydht usage involving two MyDHT instances.

So the comminucation of a single broadcast, no querying implemented…

The POC does not include any kind of error management (user should send its striple of error) or queries when content is missing : if something goes wrong it panics.

In the POC we start with existing configuration, notably the voteconf with the list of votant.

Poc target

Reaching this line (using run_multi_peer.sh script) of src/service.rs :

println!("Find my vote and received all votes!!!!!");

At this line my client did send and find all envelope, participation and vote. All those those are validated and my clients own element were recognized.

AnoDHT

The POC send its message using two different instances of MyDHT, the first for normal exchanges (maindht) and the second for sending message through tunnels. Tunnels building and proxying is done with ‘mydht-tunnel’ default implementation (mydht tunnel trait conf similar to a mydht trait conf still need to be implemented) in a second myDHT instance.

Tunnel

The limiter use by the tunnel is the limiter from ‘sized_windows_lim’ crate. This was a good choice as it let me find some bugs with some combination of openssl eas padding, the size of the limiter and the read buffer size (related to proxying ciphered padding). It is certainly not the right limiter to use.

The symetric shadow is some eas cbc 256 from openssl, and the symetric key exchange done through some peer RSA 2048 key.

The tunnel length is 4 :

  const INIT_ROUTE_LENGTH : usize = 4;

meaning that we use only two intermediatory/proxy peers. No bias is currently defined (easier to test with a small peer pool).

Peer

The peer struct for both dht (ano and main) is the same, it is a standard RSA peer from mydht-openssl crate plus a secondary address and striple informations.

pub struct StriplePeer<A : KVContent,B : Address,C : OpenSSLConf, S : StripleKind> {
  pub inner : RSAPeer<A,B,C>,
  id : Vec<u8>,
  from : Vec<u8>,
  sig : Vec<u8>,
  pub secaddress : B,

In main.rs :

type RSAPeer = StriplePeer<String,SerSocketAddr,RSA2048SHA512AES256,Rsa2048Sha512>;

String is some random peer content, SerSocketAddr is IpV4 address, RSA2048SHA512AES256 is the conf type for the mydht peer, Rsa2048Sha512 is the striple definition for peer as a striple.

A struct ‘AnoPeer’ redefine peer traits implementation for the AnoDHT (to use the right address (not the same one as in the maindht). Note that the peer use the same key pair in mydht and anodht : that is not a good idea (.

NoAuth usage : authentication in the tunnel does not make a lot of sense and have not really been tested. At some point we should limit the allowed peers (through peer mgmt interface), and then we could try to activate it.

Service

The spawner : configured to run on threads, I do not really care for this POC.

Local/global inner tunnel service : we use the ‘mydht-tunnel’ dht basis trait, so their is no distinction between global and local service. The default ano dht implementation, it sends a value (vote or envelope) to random peer through a predefined tunnel length or if receiving it sends the value to the mainDHT (through main dht api channel). So the service is only containing a reference to the user Peer and the Maindht Api Channel and code could be resume to :

    match req {
      GlobalCommand::Distant(_opr,AnoServiceICommand(StoreAnoMsg::STOREENVELOPE(envelope))) => {
        let enveloperef = ArcRef::new(MainStoreKV::Envelope(envelope));
        let c_store_env = ApiCommand::call_service(MainKVStoreCommand::Store(KVStoreCommand::StoreLocally(enveloperef,1,None)));
        self.1.send(c_store_env)?;
        Ok(GlobalTunnelReply::NoRep)
      },
      GlobalCommand::Distant(_opr,AnoServiceICommand(StoreAnoMsg::STOREVOTE(vote))) => {
        let voteref = ArcRef::new(MainStoreKV::Vote(vote));
        let c_store_vote = ApiCommand::call_service(MainKVStoreCommand::Store(KVStoreCommand::StoreLocally(voteref,1,None)));
        self.1.send(c_store_vote)?;
        Ok(GlobalTunnelReply::NoRep)
      },
      GlobalCommand::Local(AnoServiceICommand(StoreAnoMsg::STOREENVELOPE(envelope))) => {
        // proxy message
        Ok(GlobalTunnelReply::SendCommandToRand(AnoServiceICommand(StoreAnoMsg::STOREENVELOPE(envelope))))
      },
      GlobalCommand::Local(AnoServiceICommand(StoreAnoMsg::STOREVOTE(vote))) => {
        // proxy message
        Ok(GlobalTunnelReply::SendCommandToRand(AnoServiceICommand(StoreAnoMsg::STOREVOTE(vote))))
      },
    }

Tunnel construction

The main difficulty with the tunneled message is to build a route from a set of peer that would not leak information : obviously in the POC it is not very solid. - The choice of peer is route building is inner to ‘mydht-tunnel’ : Rp default implementation of ‘RouteProvider’ trait as described in a previous post (plus an additional function to choose a random recipient). - At this time it is not trully random because we do not allow us to be in the tunnel as myDHT do not allow to manage ourselve as a peer (our peer is simply not in the pool of selectable peers), ‘Rp’ could be allowed to optionaly insert our peer at intermediatory position (still biased as it is an always reachable peer).
Another huge problematic is the connection pool (analysis of established connection leaking those peer selection) : using an unreachable user in the middle of the tunnel is bad (can be detected with error returned (not used in poc) or timeout) because it implies a retry with another route which is indeed a leak of information. So anodht should maintain a reachable user pool in a random way (in the POC the pool is of size of the route so it is realy easy to analyse by looking at anodht connections). - RandomRoute struct is use as a Routing implementation, it is useless in this case but the code should move to mydht (routing to random n peers would be nice to have), it involves an additional trait on the connected peer cache : ‘IndexableWriteCache’.

As say before, connected peer pool is currently only build from conf (json containing other peers info). The connection is done through anodht peer pool size adjustment.

Next thing to do for anodht peer pool management (before trying to maintain some pool) : - activate PeerStatusListener on maindht global - on peeronline message in maindht global : send message to add/connect peer to anodht (a GlobalTunnelCommand::NewOnline), idem for offline. That way we do not double manage our peer (same peer type basis for both dht). - remove peer from anodht peerstore (peerstore should be disable by using a NoSpawn and disabling peer discovery for anodht) - when pool from anodht is to small to build route, do not send the PoolSize modification to the andodht but to the maindht instead (need to change SendCommandToRand to callback to inner service on missing peer). This is very important as currently if initial peers do not have the right address, it can not change (or anodht interpeer message does not allow peer query while maindht can), so at the time the anodht pool is very static we simply try to connect. It also makes the nat traversal a bit useless.

MainDHT

Peer

Same StriplePeer struct as before but with native traits implementation (not using AnoPeer).

Public authentication scheme : openssl default auth shadower is only usable in private mode so it is disabled for StriplePeer.

Use of openssl message shadower by using common RSAPeer keys.

Service

Spawner : configured to run on threads

Protocol implemented in global service, so no local service usage.

The implementation of the protocol, it is a bit messy and should be refactor (lot of useless clone due to an initial filtering idea of the inner KVStore service message).

Protocol steps are chained in the global service, and involve : - exchange messages and validation. - broadcast publicly to all vote peers (either our message or a message from another unknown peer received in anodht (vote or envelope)). - send to a random peer through a tunnel by using the anodht instance. - validate content and chain task when condition reached, there is striple validation but also two hand written one : - commitment of envelope matching its vote revealed commitment parameters. - participation envelope set identique between all participation messages (currently simply check of all ids, a simple merkle tree root hash would be more suitable).

For instance:

// make vote (sign by enveloppe, about votedesc)
let vote = Vote::new(&(&context.my_envelope,&context.my_envelope_priv_key), &context.vote_desc, context.my_reply.clone())?;

//assert!(vote.check(&context.my_envelope).map_err(|e|StripleMydhtErr(e))?);

context.my_vote = Some(vote.clone());
// share votes (store + query all) in anonymous dht
let c_store_vote = GlobalTunnelCommand::Inner(AnoServiceICommand(StoreAnoMsg::STOREVOTE(vote)));

//send ano vote
let command = ApiCommand::call_service(c_store_vote);
self.ano_dhtin.send(command)?;

Note that MainDHT as been build other a standard KVStore, it was a bad choice as we do not query and only push. Yet I keep the definition as it is, most of the service code benefit from additional common tools to include to the standard kvstore service (broadcast first).

TODO

  • Error management by sending a descriptive striple to a abort vote (similar to ko participation message).
  • Simple KV discovery

Striple

Use of striple library is not really usefull, but a POC is a good place to try using it.

Lot of refactoring on the lib was done to be able make a rust struct a striple simply (initially a struct would need to be converted to a simple striple implementation), and some additional striple trait where created. Mainly those three traits implementation :


impl StripleImpl for VoteDesc {
  type Kind = PubSha512;
}
impl StripleFieldsIf for VoteDesc {
  #[inline]
  fn get_algo_key(&self) -> ByteSlice {
...

impl InstantiableStripleImpl for VoteDesc {
...

Here VoteDesc struct could be use as a public sha 512 striple.

So for all aspect of signing between messages, striple abstraction is used

Common striple are load on start, yet a full descriptive striple context was not defined and most of the striple got wrong or unprecise descriptive (about) striple attached.

lazy_static!{
pub static ref STRIPLEREFS : StripleRefs = init_striple_refs().unwrap();
}

This is a good illustration of striple use, it makes reasoning easy : for a client execution of the protocol, we reach the following striple action in service.rs

// an envelope sign by (public signing) the vote desc:
let (envelope,my_envelope_priv_key) = Envelope::new(&vote_desc, my_commitment)?;

// check envelope received (right vote)
let valid_env = envelope.check(&context.vote_desc).map_err(|e|StripleMydhtErr(e))?;

// a participation signed by user
Participation::new(
  &(me, &self.me_sign_key[..]),
  &context.vote_desc,
  &context.envelopes,
  valid
)?

// check participation receive with for the corresponding user
let fp : &P = from.user.borrow();
let valid_p = participation.check(fp).map_err(|e|StripleMydhtErr(e))?

// vote striple signed by envelope
let vote = Vote::new(&(&context.my_envelope,&context.my_envelope_priv_key), &context.vote_desc, context.my_reply.clone(), context.my_commit_nonce.clone())?;

// check received vote with existing enveloppes
let valid_vote = is_commit_ok && vote.check(&context.envelopes[env_pos].0).map_err(|e|StripleMydhtErr(e))?;

TODO

  • BCont from striple is very wrong as it does not allow inline usage of stream : a Reader returned by the striple would be best (same with the multiple ids of content which would be best as an iterator : allowing bug numbers of relation backed for instance by a db query).
  • use of a serde fork for initialization method, got to switch to the alternative approach when it will be completed (see this pull).
  • Communication between both DHT is using the standard external api. Some cross service channel could be use (especially anodht using a weak channel to maindht global service).

MyDHT notable lib change

  • query peer from global service Similar to the way global service can ask mainloop for a connection to an address.
  • weak channels with handle between global service and peer store
  • fix shadower to allow some unread padding

MyDHT roadmap?

Still missing things : - networking pattern like pub/sub or broadcast : initial use of kvstore service was a bit naive and a broadcast was written in the service. - Query for changed/updated/different service return value with primitive to filter the right ones (initial mydht was for immutable keyval but we extend to service query reply). - peer discovery : currently peer discovery occurs but it limits to the peerstore values -> a configurable peer discovery is really needed (requires previous patterns) to automatically query peers (those query are accessible from outside) and look for the right ones (need to allow . At the time it could only be done manually by application through the api. - timer - peer connection management (when timer implemented we could close regularily some peers). Notably the use case where we close the transport and keep the service state and restore later : allowing big available authenticated peer pools, in this case restore of connection requires some service state to check (with maindht shadower it is useless but would be needed with a no secret shadower) for avoiding authentication rerun. - Perfect Forward Secrecy (PFS) for key exchange in mydht auth requires some minor adjustment (number of query is already fine and challenge could be use as a vector but we need to keep trace of it in our client and server service and use it as input for message shadower -> need to slightly change some traits). - Macro V2 to derive and compose : obviously not really related to this POC - Default bootstrap macro to avoid defining all myDhtconf subtype and implementation, but just exception : lot of copy paste when defining the various hashmap or spawner. - peer pool are currently authenticated and connected peer, to scale we need to maintain some reachable authenticated peer pool (no client/server service but an authentication state to possibly lighten authentication on next connection) this is complementary with previous poin about peer connection management. - logging is lacking a lot, using a dedicated crate seems quite important (maybe slog), similarily myDHT should include some optional trait interface for metrics.

The path towards something stable is still a long one, but writing this POC does point out some really needed functionalities for MyDHT.

Yet this POC and this post is a nice way to check mydht design and its current status.

Still at this point some others very interesting things to test should go first (attemting to run myDHT on webassembly).