KansasCity/July2012Hack: Difference between revisions

From My Wiki
Jump to navigation Jump to search
← Older edit
imported>Bnewbold
No edit summary
imported>Bnewbold
links, paste in email
 
(5 intermediate revisions by 2 users not shown)
Line 1: Line 1:
A few people will be traveling through Kansas City in early July, 2012, to hack on FNF projects.
A few people traveled through Kansas City in early July, 2012 to hack on FNF projects. See also:
 
* [[KansasCity/July2012SiteSurveys]] for locations and radio throughput
* [http://etherpad.freenetworkfoundation.org/p/july_sprint july_sprint etherpad]
* TODO: uploaded photos
= Narrative =
 
''(copied from a [http://freenetworkfoundation.org/pipermail/builders_freenetworkfoundation.org/2012-July/000850.html builders mailing list update])''
 
We have tested and are temporarily deploying a "Distribution" network (a
wireless network between a radio connected to a FreedomLink in a data
center and a set of FreedomTowers) using Ubiquity radios in a transparent
bridge mode. Specifically, wireless distribution system (WDS) over the
proprietary AirOs layer 2 protocol. This might be a configuration we
recommend for small scale deployments, but will probably not work well
with more than a handful of nodes. We are not happy on principle using a
proprietary protocol (replacing 802.11 to get better point-to-point
performance), even at a non-visible lower level; recommendations of open
alternatives to 802.11 would be great. Definitely a discussion of
principles to be had. The major change at this network level is moving
away from a "mesh" or mobile ad-hoc system towards something more managed;
in a larger deployment static addressing of Towers and a "regular" routing
protocol like OSPF might be appropriate here. To re-iterate, any
recommendations we come up with are just recommendations and communities
will of course select the technologies most appropriate for their
situation.
 
At the "Access level" (a wireless network between Nodes all close to a
Tower), we have been trying to get a level 3 mobile ad hoc network
("mesh", or less ambiguously, "MANET") set up between a handful of
single-band Ubiquity NanoStations and a dual-band Netgear WNDR3800 router.
The plan was to use commotion, which is an OpenWrt-based router
distribution which aims to make configuring OLSR mesh networks easy.
Commotion is still under heavy development and images weren't available
for the WNDR3800, so we're now experimenting with the new qmp ("Quick Mesh
Project") firmware, which is is also OpenWrt based, and can use OLSR, but
by default uses the "BMX6" protocol, for which documentation is scarce. We
are skeptical of fracturing to yet another firmware distro, but have been
very pleased with qmp so far. It seems to have some support by both the
FreiFunk and guifi.net communities, and is under development right now. We
have had some trouble getting the "community network" mesh mode working,
but the "roaming" configuration worked out of the box:
 
  http://qmp.cat/projects/qmp
  http://qmp.cat/projects/qmp/wiki/Roaming-Collaborative
 
qmp's "community" numbering scheme is interesting: all nodes get proper
IPv6 addresses which are used in all mesh routing and calculations, while
IPv4 addresses are tunneled from end points to the uplink/gateway servers,
which do NAT. this aligns with goals for encrypted end-to-end tunneling
and the reality of limited access to "real" (globally routed) IPv4
numbers. Food for thought, particularly as centralized IPv4 NAT gateways
are an obvious target for connection-level surveillance.
 
Much work to be done at the Access network layer; a layer 2 protocol like
BATMAN-adv might be more appropriate, or Babel, or any one of dozens of
other mesh configurations. A top selection metric is having an up-to-date,
almost-plug-and-play router distribution. Again,
participant-mileage-will-vary and communities are totally free to use the
routing protocol flavor of the month if they want.
 
Just to reiterate and set context one last time, this is just experimental
work to shake out unknown unknowns and field test new technologies. The
tools and architecture we are using do no necessarily meet all of the
FNF's technical and ideological goals, and should not be interpreted yet
as the consensus plan or official recommendations.
 
 
==== Tuesday July 10th, 2012 ====
 
Made plans for a dual FreedomTower arrangement with a single radio uplink, with
point to point links between all three stations. Didn't have enough equipment
to additionally build out Commotion-based mesh networks off each FreedomTower,
so we went shopping for several slow hours at Microcenter (an
electronics/computer department store). Even using openwrt.org and wikidevi.com
wikis from in-store computers it was hard to find fully supported equipment
that was in stock and at a reasonable price, even though the available
selection was large.
 
In the afternoon we spent several hours getting new equipment working, with
limited success. Linux device driver support is pretty good these days, but
they are frequently non-free or not in the default installation.  Lesson
(re)learned: use "consensus" hardware and order it from the online stores or
ebay.
 
We installed pfSense on a second foxconn nettop computer (requiring monitor
and keyboard), and compiled OpenWrt from scratch to see how that process goes
(pretty smoothly!), thinking we might have to rebuild the Commotion firmware
built on top of OpenWrt at some point to support new hardware platforms. We
adopted the Transit, Distribution, Access, Edge terminology
 
==== Wednesday ====
 
Wednesday (July 4th) we wrapped up fiddling with wireless devices and
compilation and started to configure the Commotion wireless firmware on
Ubiquity NanoStations for testing. While the Commotion user interface is slick,
it still seems to be at an early phase of development/release and it was hard
to find up to date images to flash to our hardware. We read up a bit on OLSR
and considered other MANET protocols (none of us have a deep understanding) in
hopes of finding a plug-and-play solution. Eventually we stumbled upon the
promising qmp (Quick Mesh Project) firmware. qmp worked first try in roaming
mode but we had confusion and trouble setting up "community" mode, which was
closer to the architecture and numbering scheme we wanted.
 
We configured and tested the 3.65GHz NanoStations in a transparent bridge mode
(using WiFi WDS), discussed numbering strategy, and spent a good deal of time
fiddling with 192.168.0.0/16 local subnets (should have just used 10.0.0.0/8
from the start).
 
By this point we decided to reduce the scope of our test system to a single
data center to FreedomTower connection with a qmp mesh behind that; this
allowed us to use NanoStations for the mesh.
 
==== Thursday ====
 
We visited the Oak Tower data center, checked out lines of sight, and installed
the access point end of our 3.65GHz point-to-point link (without working uplink
at this time), and then confirmed link-level connectivity from an outdoors site
just a couple blocks away.
 
Thursday we continued experimenting with qmp a bit, including running a build
for the Netgear WNDR3800 router hardware.
 
We also made a brief visit to the Hammerspace hackerspace in Kansas City, where
Isaac gave a quick talk introducing the FNF.
 
==== Friday ====
 
We got some very helpful feedback from the qmp developers (via mailing list),
which helped us choose functioning settings for the "community" mode of qmp
(still needed to restart bmx6 manually on the nodes themselves after reboot).
 
In the afternoon we drove around Kansas City checking line of sight and
link-level connectivity and throughput to the Oak Tower radio (which still did
not have uplink). In the evening we returned to the Oak Tower data center and
after a few hours of crude hacking got the 3.65GHz radio link functioning as a
layer 2 bridge to the main FNF router, which provided DHCPv4 and NAT to the
internet as a temporary work around.
 
==== Saturday ====
 
On Saturday morning we only had an hour or two of free time, but we were able
to drive out and successfully test internet connectivity at a park location
several kilometers from Oak Tower. We used the 3.65GHz link plus a single-hop
qmp community mode mesh to a laptop. There wasn't time to set up more hops,
align the radios for a best-case bandwidth test, or attach a FreedomBox at the
edge, but we were very excited to confirm that the link worked at this
distance, easily streaming HTML5 video from http://freenetworkfoundation.org!
 
= Notes =
 
For commotion-specific olsrd config settings, see:
https://code.commotionwireless.net/projects/commotion/wiki/Olsrd_Configuration_Options
 
Considered alternatives to commotion (openwrt-based OLSR distributions):
* qmp ("quick mesh project")
* fabfi: heavy dev on v5, not stable? "double layer" mesh, all IPv6 with IPv4 tunneled, nice design
* freifunk: old, no easily found new distro
* ro.b.in/OLSR "original version": old, little info (vs. BATMAN version)
* Robin2: nanostations only? http://wiki-robin.meshroot.com/Background
 
= Pre-planning =


== Logistics ==
== Logistics ==
Line 6: Line 168:


Trip to open ecology farm would be most excellent!
Trip to open ecology farm would be most excellent!
KC Hammerspace hackerspace has an open house (6pm) and quick talks(7:30pm) thursday


== Goals ==
== Goals ==
Line 42: Line 206:
** buy some freedomstations?
** buy some freedomstations?


* extras
* extras << DC locker has most/all of below items.
** ethernet patch cables
** ethernet patch cables
** power strips
** power strips
Line 51: Line 215:


== TODO ==
== TODO ==
* choose test sites for freedomtower, nodes
* choose test sites for freedomtower, nodes << Isaac handling. http://chili.freenetworkfoundation.org/issues/281 and http://chili.freenetworkfoundation.org/issues/282
* schedule visit to Joe's
* schedule visit to Joe's  
* order any necessary hardware
* order any necessary hardware << 3.65ghz radios ordered
* plan/spec for continuous integration stuff
* plan/spec for continuous integration stuff << underway. Not currently ticketed.


== Measurements/Benchmarks ==
== Measurements/Benchmarks ==

Latest revision as of 23:38, 8 July 2012

A few people traveled through Kansas City in early July, 2012 to hack on FNF projects. See also:

Narrative

(copied from a builders mailing list update)

We have tested and are temporarily deploying a "Distribution" network (a wireless network between a radio connected to a FreedomLink in a data center and a set of FreedomTowers) using Ubiquity radios in a transparent bridge mode. Specifically, wireless distribution system (WDS) over the proprietary AirOs layer 2 protocol. This might be a configuration we recommend for small scale deployments, but will probably not work well with more than a handful of nodes. We are not happy on principle using a proprietary protocol (replacing 802.11 to get better point-to-point performance), even at a non-visible lower level; recommendations of open alternatives to 802.11 would be great. Definitely a discussion of principles to be had. The major change at this network level is moving away from a "mesh" or mobile ad-hoc system towards something more managed; in a larger deployment static addressing of Towers and a "regular" routing protocol like OSPF might be appropriate here. To re-iterate, any recommendations we come up with are just recommendations and communities will of course select the technologies most appropriate for their situation.

At the "Access level" (a wireless network between Nodes all close to a Tower), we have been trying to get a level 3 mobile ad hoc network ("mesh", or less ambiguously, "MANET") set up between a handful of single-band Ubiquity NanoStations and a dual-band Netgear WNDR3800 router. The plan was to use commotion, which is an OpenWrt-based router distribution which aims to make configuring OLSR mesh networks easy. Commotion is still under heavy development and images weren't available for the WNDR3800, so we're now experimenting with the new qmp ("Quick Mesh Project") firmware, which is is also OpenWrt based, and can use OLSR, but by default uses the "BMX6" protocol, for which documentation is scarce. We are skeptical of fracturing to yet another firmware distro, but have been very pleased with qmp so far. It seems to have some support by both the FreiFunk and guifi.net communities, and is under development right now. We have had some trouble getting the "community network" mesh mode working, but the "roaming" configuration worked out of the box: 

  http://qmp.cat/projects/qmp
  http://qmp.cat/projects/qmp/wiki/Roaming-Collaborative

qmp's "community" numbering scheme is interesting: all nodes get proper IPv6 addresses which are used in all mesh routing and calculations, while IPv4 addresses are tunneled from end points to the uplink/gateway servers, which do NAT. this aligns with goals for encrypted end-to-end tunneling and the reality of limited access to "real" (globally routed) IPv4 numbers. Food for thought, particularly as centralized IPv4 NAT gateways are an obvious target for connection-level surveillance.

Much work to be done at the Access network layer; a layer 2 protocol like BATMAN-adv might be more appropriate, or Babel, or any one of dozens of other mesh configurations. A top selection metric is having an up-to-date, almost-plug-and-play router distribution. Again, participant-mileage-will-vary and communities are totally free to use the routing protocol flavor of the month if they want.

Just to reiterate and set context one last time, this is just experimental work to shake out unknown unknowns and field test new technologies. The tools and architecture we are using do no necessarily meet all of the FNF's technical and ideological goals, and should not be interpreted yet as the consensus plan or official recommendations.


Tuesday July 10th, 2012

Made plans for a dual FreedomTower arrangement with a single radio uplink, with point to point links between all three stations. Didn't have enough equipment to additionally build out Commotion-based mesh networks off each FreedomTower, so we went shopping for several slow hours at Microcenter (an electronics/computer department store). Even using openwrt.org and wikidevi.com wikis from in-store computers it was hard to find fully supported equipment that was in stock and at a reasonable price, even though the available selection was large.

In the afternoon we spent several hours getting new equipment working, with limited success. Linux device driver support is pretty good these days, but they are frequently non-free or not in the default installation. Lesson (re)learned: use "consensus" hardware and order it from the online stores or ebay.

We installed pfSense on a second foxconn nettop computer (requiring monitor and keyboard), and compiled OpenWrt from scratch to see how that process goes (pretty smoothly!), thinking we might have to rebuild the Commotion firmware built on top of OpenWrt at some point to support new hardware platforms. We adopted the Transit, Distribution, Access, Edge terminology

Wednesday

Wednesday (July 4th) we wrapped up fiddling with wireless devices and compilation and started to configure the Commotion wireless firmware on Ubiquity NanoStations for testing. While the Commotion user interface is slick, it still seems to be at an early phase of development/release and it was hard to find up to date images to flash to our hardware. We read up a bit on OLSR and considered other MANET protocols (none of us have a deep understanding) in hopes of finding a plug-and-play solution. Eventually we stumbled upon the promising qmp (Quick Mesh Project) firmware. qmp worked first try in roaming mode but we had confusion and trouble setting up "community" mode, which was closer to the architecture and numbering scheme we wanted.

We configured and tested the 3.65GHz NanoStations in a transparent bridge mode (using WiFi WDS), discussed numbering strategy, and spent a good deal of time fiddling with 192.168.0.0/16 local subnets (should have just used 10.0.0.0/8 from the start).

By this point we decided to reduce the scope of our test system to a single data center to FreedomTower connection with a qmp mesh behind that; this allowed us to use NanoStations for the mesh.

Thursday

We visited the Oak Tower data center, checked out lines of sight, and installed the access point end of our 3.65GHz point-to-point link (without working uplink at this time), and then confirmed link-level connectivity from an outdoors site just a couple blocks away.

Thursday we continued experimenting with qmp a bit, including running a build for the Netgear WNDR3800 router hardware.

We also made a brief visit to the Hammerspace hackerspace in Kansas City, where Isaac gave a quick talk introducing the FNF.

Friday

We got some very helpful feedback from the qmp developers (via mailing list), which helped us choose functioning settings for the "community" mode of qmp (still needed to restart bmx6 manually on the nodes themselves after reboot).

In the afternoon we drove around Kansas City checking line of sight and link-level connectivity and throughput to the Oak Tower radio (which still did not have uplink). In the evening we returned to the Oak Tower data center and after a few hours of crude hacking got the 3.65GHz radio link functioning as a layer 2 bridge to the main FNF router, which provided DHCPv4 and NAT to the internet as a temporary work around.

Saturday

On Saturday morning we only had an hour or two of free time, but we were able to drive out and successfully test internet connectivity at a park location several kilometers from Oak Tower. We used the 3.65GHz link plus a single-hop qmp community mode mesh to a laptop. There wasn't time to set up more hops, align the radios for a best-case bandwidth test, or attach a FreedomBox at the edge, but we were very excited to confirm that the link worked at this distance, easily streaming HTML5 video from http://freenetworkfoundation.org!

Notes

For commotion-specific olsrd config settings, see: https://code.commotionwireless.net/projects/commotion/wiki/Olsrd_Configuration_Options

Considered alternatives to commotion (openwrt-based OLSR distributions):

  • qmp ("quick mesh project")
  • fabfi: heavy dev on v5, not stable? "double layer" mesh, all IPv6 with IPv4 tunneled, nice design
  • freifunk: old, no easily found new distro
  • ro.b.in/OLSR "original version": old, little info (vs. BATMAN version)
  • Robin2: nanostations only? http://wiki-robin.meshroot.com/Background

Pre-planning

Logistics

B+I arrive Tuesday July 3rd, B out Saturday the 7th, C shows up Saturday the 7th (?).

Trip to open ecology farm would be most excellent!

KC Hammerspace hackerspace has an open house (6pm) and quick talks(7:30pm) thursday

Goals

  • FreedomLink - FreedomTower - FreedomNode - FreedomBox demo
  • experiment with mesh wifi distributions: byzantium, commotion, generic batman/olsr, open-mesh, etc
  • FreedomLink - FreedomTower - FreedomTower test (aka, 1 backhaul hop)
  • plan + implement continuous integration for FreedomLab?
  • multi-homed FreedomTower: FreedomLink + 4G?
  • configure routing through IPv6 block at KC DC?
  • try out ChaosVPN?

Hardware

  • radio link betwen KC DC and freedomtower: directional 5GHz?
    • plus PoE, cabling, mounting hardware
  • router/firewall inside KC DC
  • complete FreedomTower
    • plus radio link to KC DC
    • if any new "recommended" router hardware, test with that
    • power management device C recommended
    • if multi-homed, need 4G modem and service plan
  • FreedomNode
    • should really test at least 2x nodes with a mesh hop?
    • b has a routerstation pro; only 1x radio card
    • rig up something using nanostations?
  • FreedomBox
    • b and i both have dreamplugs, also have extra microSD cards
  • smartphones and laptops as end user devices
  • "out of band" 4G or 3G data access for debugging and configuration?
  • any extra routers for mesh experimentation
    • commodity plus OpenWRT? (b has a few)
    • routerstation pro (b has one)
    • dreamplugs (b+i)
    • TP-Link mini devices? (fnf has some?)
    • buy some freedomstations?
  • extras << DC locker has most/all of below items.
    • ethernet patch cables
    • power strips
    • cable ties
    • mounting hardware
    • enclosures
    • watts up (b has one)

TODO

Measurements/Benchmarks

  • sustained throughput, packets/sec, connections, latency per link
  • power cycle any equipment, see that system resumes correctly
  • energy usage of all equipment individual and combined (max, idle, typical)
Retrieved from "https://commons.thefnf.net/index.php?title=KansasCity/July2012Hack&oldid=1835"