From 3c769a1e9329814023b603cdceaba10dbabf804b Mon Sep 17 00:00:00 2001 From: Andrew Kesterson Date: Sun, 19 Aug 2012 08:06:23 -0400 Subject: [PATCH] Moving things out of the README into the wiki --- README.md | 215 ++---------------------------------------------------- 1 file changed, 8 insertions(+), 207 deletions(-) diff --git a/README.md b/README.md index 6fbf9a8..d6dc570 100644 --- a/README.md +++ b/README.md @@ -3,215 +3,16 @@ disco Dead Simple COnfiguration management and continuous integration for linux like systems -DISCO is just now beginning development. Expect what you find here to do irreparable damage -to any system you run it on. I ONLY run disco on a throwaway VM at current. +DISCO uses simple, proven technologies to achieve the goal of configuration management. -Why disco? -===== +* rsync for data transfer from master to client +* bash for scripts and templating +* unionfs and restricted bash for noop operations +* plaintext files in a structured directory tree for node classification and configuration -Because puppet, chef, cfengine, etc, are all great tools, but they all fall -short of the mark, in terms of simplicity, ease of use, and reliability. -None of them really follow the UNIX philosophy of "do one thing, do it well, -and don't reinvent the wheel". - -No, really, why did you name it "disco"? -===== -I wanted an acronym based off of "Dead Simple Continuous Integration", and this was -the closest I found. - -Requirements -===== - -DISCO assumes that you: - - - have at least one server capable of running rsyncd, sshd and a recent GNU - - have one or more clients capable of running ssh, rsync, fuse-unionfs, and recent GNU - -While that's a very simple requirements list, it currently restricts it to recent Linux -systems. You may or may not be able to use this tool on FreeBSD or Mac OS X, I haven't -tried. Due to the way it executes, this tool will probably never, ever execute properly -on Windows. - -Installing DISCO -===== - - 1. Check out the sources - 2. sudo make install - -Why focus so much on linux? -===== - -Because if we try to do everything and the kitchen sink, for every OS out there, we run -the risk of falling short in the same ways the other CI tools have. By limiting our scope -and problem space to recent GNU/Linux systems, we can write a much simpler tool in a -much shorter amount of time that is much simpler to understand. - -Is DISCO noop friendly (report all incoming changes)? -===== - -Yes, DISCO is noop friendly, with a caveat: The way we implement noop is through restricted bash -shells. This is generally sufficient, and already proven and simple. - -There are some questions around "is the NOOP really secure then?" Well, yes and no. -The disco NOOP, like any (bash --restricted) shell, can be broken out of without a whole lot of work. -Especially considering that the restricted shell runs as root. In this case, the disco NOOP, -much like a barbed wire fence at a buffalo farm, is not expected to ACTUALLY keep anyone inside - -it's just a gentle suggestion that you please not run all over the farm breaking things. - -Unlike puppet's noop, which is implemented via a guaranteed safe DSL, DISCO assumes an -existing trust network between your disco server and disco client; the goal of DISCO noop is to -prevent well-meaning trusted sysadmins from accidentally doing stupid things. It does not try -to secure your systems from malicious code. That security layer is moved up, onto the maintainer, -who must verify the sanity of all code they are sending to client machines. - -Specifically: Since DISCO use a unionfs and chroot for the restricted bash shell, I am confiden -that your physical disk is safe from accidental modifications (everything is captured on the -scratchpad, with the exception of writes to /proc and /sys, which are read-only and simply discarded). -The bit that I can't promise is that one of your module maintainers won't find a way to kill a -running process, or signal/restart a service in a way that DISCO can't trap/log/noop it. -This may or may not be a big deal to you, but should be a consideration before you start -migrating to DISCO - how much do you trust your module maintainers? - - -How do you establish the trust relationship? -===== - -DISCO uses rsync(+ssh) with SSH keys, so the answer is, "we don't establish it" - SSH handles -that for us by the server allowing or denying the key. - -How do you handle parameters (like puppet ENC, etc)? -===== - -DISCO uses a section of the filesystem to layout a tree of pathable, walkable parameters. -This part of the filesystem is available to the client at execution time, so these variables -can be used in scripts, templates, and definition files, to further customize execution based -off of parameters. This lives on the SERVER, not the client. - -From the server perspective, the parameters tree looks like: - - /var/disco/parameters - ___ ___ disco - ___ ___ ___ client - ___ ___ ___ ___ cmds - ___ ___ ___ ___ ___ rsync - ___ ___ disco - ___ ___ ___ server - ___ ___ ___ ___ uri - ___ ___ NODE_NAME - ___ ___ ___ modules - ___ ___ ___ ___ ... - ___ ___ ___ parameters - ___ ___ ___ ___ ... - -Think of it like a large JSON document expressed as a filesystem, with the document keys the -filenames, and the values being their contents. This format was chosen because it can be easily -created from any number of other existing datasources, and doesn't tie DISCO to any one particular -tool (cobbler, etc). The admin is free to create this structure on the server however they please. - -Given this, disco does not use a config file, all configuration parameters are present in this -tree. - -There are only two possible toplevel paths, /disco and /NODE_NAME. NODE_NAME is equal to the -FQDN of the client making a request, and /disco is the internal client/server configuration. -The parameter tree is transmitted from the server to the client via (yet another) rsync -operation, and is accessible as a filesystem tree (or the disco-param command which is just a -bash wrapper). These parameters appear in /var/disco/parameters on the client and server, and -default values can be found there in the client/server install before the first run of the client. - - disco/client/cmds/rsync : The rsync command to use when synching - files. - disco/server/uri : The rsync URI from which to fetch module definitions. - disco/NODE_NAME/modules : This list defines the modules to install - on a given node. - disco/NODE_NAME/parameters : This tree defines all configuration - parameters for the node not related to any module in particular. - -Some special parameters are provided to the client, that do not exist on the paramters tree until -runtime: - - disco/NODE_NAME/current_module : This parameter defines the full - name of the current module, such that a module definition file - can access its personal parameters via without knowing its name, e.g.: - $(disco-param get $(hostname)/classes/$(disco-param get $(hostname)/current_module)/some/module/specific/path) - -How to deploy stuff -===== - -DISCO uses rsync(+ssh) to distribute files, and bash to execute supporting scripts. It has a -rudimentary dependency mechanism implemented via a topological sort. - -Essentially, to deploy something, you need 3 things: - - - some static files on an rsync server - - some bash templates (scripts that output what their content should be) on an rsync server - - some scripts that may or may not do something with those files and templates - -Scripts -===== - -DISCO uses bash for a scripting and templating engine. Instead of writing a custom DSL that lets -you specify operations (like Puppet did) or utilize a higher level language (like Chef did with -ruby), DISCO just uses the proven bash shell. - -Every time your module is executed (e.g. every time the disco client executes), all of the scripts -are executed. The order of execution is determined by alphabetically sorting the filenames, rc.d -style; so naming your scripts as 00-fix_perms.sh, 10-correct_nodes.sh, etc, will cause them to be -executed in the proper order. This prevents you from having to create a separate file that describes -the execution order. - -Files vs Templates -===== - -Files and Templates are delivered exactly the same way - via rsync. - -Files are static files who are delivered on to the disk, and no more operations are done to them. -They are delivered with the same permissions that they are given by the rsync repository. - -Templates are bash scripts who are delivered on to the disk, and then they are executed, with their -file contents replaced by their output. Templates are subject to all the same restrictions as scripts -(be mindful of the constraints of $NOOP), and in addition, they are ALWAYS interpolated in the safe -NOOP execution environment (file modifications will be discarded, and only rudimentary bash builtins -are enabled). Templates have access to all client parameters via the disco-param command. Templates -will end up with the same permissions that rsync gives them. - -Module Layout -===== - -A disco module (also called a "disco ball" for fun) looks like this: - - MODULE - ___ requires - ___ parameters - ├── files/ - ├── scripts/ - └── templates/ - -Your module can theoretically pull files, scripts, and templates from any location that can be -reached via rsync; however, it is generally considerd good form to include all things relevant -to your module, inside its disco ball. The disco ball is then placed in an accessible location -on the rsync server, and the disco client will pull all modules, files, scripts, and templates -relevant to its execution, and run them. - -ALL MODULE FILES, SCRIPTS, AND TEMPLATES ARE DELIVERED RELATIVE TO / ON THE CLIENT. - -MODULE/requires -===== - -This file lists, one name per line, the names of other modules that must be installed on this -node in order for this module to install correctly. This is used to create a dependency graph, -and thereby determine execution order. - -This file is optional. - -MODULE/parameters -===== - -Each module can define default parameters which will be made available to all clients using the module. -These parameters will be merged together on the client at module fetch time, and any node-specific -parameters will override any default parameters specified here (they are rsync'ed over the top of each -other). These parameters will be rooted at /MODULE_NAME/... . - -This tree is optional. +DISCO is in the very early alpha stages of development. At this point, I am fairly confident +that it won't destroy the system you run it on, but I still only run it on disposable virtual +machines until it reaches a higher level of maturity. Server Side Setup =====