Hostsurvey

Hostsurvey is the tool that connects Bmd users. Experienced database users could consider it a specialized replication tool. Those familiar with web hosting can think of it as a semi-automatic mirroring system. Its goal is to copy any changes from one instance of a database to all the others. For a popular database, that could be a lot of instances on a lot of machines. I don't have a lot of bandwidth to play with here, so the task has to be very decentralized. Here's the model: everyone has a copy of the hostsurvey software and runs it regularly to monitor changes on several other users' instances, but no one has to monitor everyone. As long as there is a path from everyone to everyone (fully connected network) and the many hostsurveys run regularly, then the massive replication job should solve itself, albeit slowly.

Now let's get technical

Hostsurvey is just a big perl script that should run all the time. Right now, you probably should run it as your own user account name, but in the future it will run as "bmd". We'll create an application account like that later. Just remember it needs write access to all your database table files (.tbl's), sync files (.sync-date), the memberdirectory itself, and possibly more one day. It will spend most of its time sleeping in wait for the next nextchktime. The hostlist table in every Bmd database serves as a configuration table for hostsurvey. Here's a breakdown of the fields.

nickname

The nickname is mostly for human reference, but is also used to name a temporary directory in /usr/bmd/tmp. After running hostsurvey, you can find a /usr/bmd/tmp/(nickname) directory for every instance that was checked.

indexfile

This field needs to contain the fully qualified url of a host's indexfile. Only the indexfile name is necessary to open a database assuming everything else is setup correctly.

lastmtime

This was the modification time of your host's indexfile, the last time hostsurvey checked it. It may change every time that instance is checked. This is used to avoid downloading more than one day worth of changes. All times are in epoch relative form, though they may not always look like it.

nextchktime

This is the next time hostsurvey plans on checking a particular host's instance for changes. It should increase after each check or you could change it by hand.

interval

This is a time interval between checks. Hostsurvey normally checks each host's database every day which is 86,400 seconds, but you could change it to whatever you want.

Hostsurvey will wait for the designated time to check hosts while watching your hostlist table files for changes every five minutes. It will download the host's sync files for all the days since the last modification time listed in the hostlist's lastmtime field and store them in /usr/bmd/tmp/(nickname). If the lastmtime field is blank, then hostsurvey must do a "full compare" which involves downloading the host's entire database. Obviously, we'd prefer to avoid that to save bandwidth.

Once all necessary sync files are stored on your harddrive, hostsurvey can begin adding the changes to your own database. This is the part where hostsurvey needs write access to your table files, sync files and to their member directories. This part is also going to change in the future. I would prefer to give each user some approval control over the content of his own instance before those changes can potentially spread to others. This means hostsurvey is going to split into several files so some of its capabilities can be included into Jester. Why don't you read about Jester next.