

Developers Guide to the Code
----------------------------

At a high level, this code does both censuses: walks all 2^32 IP addresses;
and surveys: repeatedly probes a pre-calculated set of ip addresses.

- actually, it does this in pseudorandom order not sequentially
  see ip_iterator.{cc,hh} for how

  random_order.{cc,hh} takes care of dividing space among probers
  (e.g. this is N-th prober out of M total) using hashing.  It also
  takes care of loading/saving state, provides white/black list
  capabilities; progress report; gives common interface for cycling
  over whole ip address space using iterator or going over ip list.

  (caida_iterator.c is another implementation that is not really part of icmptrain)

- it probes as fast is it can, but with with rate limiting if
  necessary (particularly for surveys).  See icmptrain.cc for command
  options controlling that and icmp_probe.{cc,hh} for implementation
  (class ICMP_Prober).


- probes are handled carefully (i.e., cleverly) to reduce memory consumption
  -- probes are sent out with id/sequence in the payload
  -- when responses return, they're matched up with their probing address
     with a "packet matching algorithm".
     This algorithm is described at 
     http://www.isi.edu/ant/traces/topology/address_surveys/binformat_description.html
     see "Improved Packet Matching"
  -- interally, this is implemented in file icmp_probe.cc
     see ICMP_Prober::recv(), ICMP_Prober::process_echoreply(), 
     ICMP_Prober::process_unreach()


- it writes the data out in "24-byte" format, documentated at 
http://www.isi.edu/ant/traces/topology/address_surveys/binformat_description.html
  see icmiptrain_datafile.{cc,hh}

- unusual packets are saved in the pcap format, see
  icmp_probe.cc:ICMP_Prober::dump_packet().


In addition to the core probing code, the directory has other stuff

- start_icmptrain_census.sh, start_icmptrain_survey.sh	
  and reload_blacklist.sh
  are support scripts

- print_datafile.cc: utility for printing contents of the binary data
  file in a humanly readable form.  This utility is also capable of
  anonymizing ip addresses (hence it uses scramble_crypt.{cc,hh}, from
  lander anonymization code).

- caida_iterator.c is a variant of ip_iterator.cc shared with CAIDA in Oct 2010
