Moving things out of the README into the wiki
This commit is contained in:
215
README.md
215
README.md
@@ -3,215 +3,16 @@ disco
|
||||
|
||||
Dead Simple COnfiguration management and continuous integration for linux like systems
|
||||
|
||||
DISCO is just now beginning development. Expect what you find here to do irreparable damage
|
||||
to any system you run it on. I ONLY run disco on a throwaway VM at current.
|
||||
DISCO uses simple, proven technologies to achieve the goal of configuration management.
|
||||
|
||||
Why disco?
|
||||
=====
|
||||
* rsync for data transfer from master to client
|
||||
* bash for scripts and templating
|
||||
* unionfs and restricted bash for noop operations
|
||||
* plaintext files in a structured directory tree for node classification and configuration
|
||||
|
||||
Because puppet, chef, cfengine, etc, are all great tools, but they all fall
|
||||
short of the mark, in terms of simplicity, ease of use, and reliability.
|
||||
None of them really follow the UNIX philosophy of "do one thing, do it well,
|
||||
and don't reinvent the wheel".
|
||||
|
||||
No, really, why did you name it "disco"?
|
||||
=====
|
||||
I wanted an acronym based off of "Dead Simple Continuous Integration", and this was
|
||||
the closest I found.
|
||||
|
||||
Requirements
|
||||
=====
|
||||
|
||||
DISCO assumes that you:
|
||||
|
||||
- have at least one server capable of running rsyncd, sshd and a recent GNU
|
||||
- have one or more clients capable of running ssh, rsync, fuse-unionfs, and recent GNU
|
||||
|
||||
While that's a very simple requirements list, it currently restricts it to recent Linux
|
||||
systems. You may or may not be able to use this tool on FreeBSD or Mac OS X, I haven't
|
||||
tried. Due to the way it executes, this tool will probably never, ever execute properly
|
||||
on Windows.
|
||||
|
||||
Installing DISCO
|
||||
=====
|
||||
|
||||
1. Check out the sources
|
||||
2. sudo make install
|
||||
|
||||
Why focus so much on linux?
|
||||
=====
|
||||
|
||||
Because if we try to do everything and the kitchen sink, for every OS out there, we run
|
||||
the risk of falling short in the same ways the other CI tools have. By limiting our scope
|
||||
and problem space to recent GNU/Linux systems, we can write a much simpler tool in a
|
||||
much shorter amount of time that is much simpler to understand.
|
||||
|
||||
Is DISCO noop friendly (report all incoming changes)?
|
||||
=====
|
||||
|
||||
Yes, DISCO is noop friendly, with a caveat: The way we implement noop is through restricted bash
|
||||
shells. This is generally sufficient, and already proven and simple.
|
||||
|
||||
There are some questions around "is the NOOP really secure then?" Well, yes and no.
|
||||
The disco NOOP, like any (bash --restricted) shell, can be broken out of without a whole lot of work.
|
||||
Especially considering that the restricted shell runs as root. In this case, the disco NOOP,
|
||||
much like a barbed wire fence at a buffalo farm, is not expected to ACTUALLY keep anyone inside -
|
||||
it's just a gentle suggestion that you please not run all over the farm breaking things.
|
||||
|
||||
Unlike puppet's noop, which is implemented via a guaranteed safe DSL, DISCO assumes an
|
||||
existing trust network between your disco server and disco client; the goal of DISCO noop is to
|
||||
prevent well-meaning trusted sysadmins from accidentally doing stupid things. It does not try
|
||||
to secure your systems from malicious code. That security layer is moved up, onto the maintainer,
|
||||
who must verify the sanity of all code they are sending to client machines.
|
||||
|
||||
Specifically: Since DISCO use a unionfs and chroot for the restricted bash shell, I am confiden
|
||||
that your physical disk is safe from accidental modifications (everything is captured on the
|
||||
scratchpad, with the exception of writes to /proc and /sys, which are read-only and simply discarded).
|
||||
The bit that I can't promise is that one of your module maintainers won't find a way to kill a
|
||||
running process, or signal/restart a service in a way that DISCO can't trap/log/noop it.
|
||||
This may or may not be a big deal to you, but should be a consideration before you start
|
||||
migrating to DISCO - how much do you trust your module maintainers?
|
||||
|
||||
|
||||
How do you establish the trust relationship?
|
||||
=====
|
||||
|
||||
DISCO uses rsync(+ssh) with SSH keys, so the answer is, "we don't establish it" - SSH handles
|
||||
that for us by the server allowing or denying the key.
|
||||
|
||||
How do you handle parameters (like puppet ENC, etc)?
|
||||
=====
|
||||
|
||||
DISCO uses a section of the filesystem to layout a tree of pathable, walkable parameters.
|
||||
This part of the filesystem is available to the client at execution time, so these variables
|
||||
can be used in scripts, templates, and definition files, to further customize execution based
|
||||
off of parameters. This lives on the SERVER, not the client.
|
||||
|
||||
From the server perspective, the parameters tree looks like:
|
||||
|
||||
/var/disco/parameters
|
||||
___ ___ disco
|
||||
___ ___ ___ client
|
||||
___ ___ ___ ___ cmds
|
||||
___ ___ ___ ___ ___ rsync
|
||||
___ ___ disco
|
||||
___ ___ ___ server
|
||||
___ ___ ___ ___ uri
|
||||
___ ___ NODE_NAME
|
||||
___ ___ ___ modules
|
||||
___ ___ ___ ___ ...
|
||||
___ ___ ___ parameters
|
||||
___ ___ ___ ___ ...
|
||||
|
||||
Think of it like a large JSON document expressed as a filesystem, with the document keys the
|
||||
filenames, and the values being their contents. This format was chosen because it can be easily
|
||||
created from any number of other existing datasources, and doesn't tie DISCO to any one particular
|
||||
tool (cobbler, etc). The admin is free to create this structure on the server however they please.
|
||||
|
||||
Given this, disco does not use a config file, all configuration parameters are present in this
|
||||
tree.
|
||||
|
||||
There are only two possible toplevel paths, /disco and /NODE_NAME. NODE_NAME is equal to the
|
||||
FQDN of the client making a request, and /disco is the internal client/server configuration.
|
||||
The parameter tree is transmitted from the server to the client via (yet another) rsync
|
||||
operation, and is accessible as a filesystem tree (or the disco-param command which is just a
|
||||
bash wrapper). These parameters appear in /var/disco/parameters on the client and server, and
|
||||
default values can be found there in the client/server install before the first run of the client.
|
||||
|
||||
disco/client/cmds/rsync : The rsync command to use when synching
|
||||
files.
|
||||
disco/server/uri : The rsync URI from which to fetch module definitions.
|
||||
disco/NODE_NAME/modules : This list defines the modules to install
|
||||
on a given node.
|
||||
disco/NODE_NAME/parameters : This tree defines all configuration
|
||||
parameters for the node not related to any module in particular.
|
||||
|
||||
Some special parameters are provided to the client, that do not exist on the paramters tree until
|
||||
runtime:
|
||||
|
||||
disco/NODE_NAME/current_module : This parameter defines the full
|
||||
name of the current module, such that a module definition file
|
||||
can access its personal parameters via without knowing its name, e.g.:
|
||||
$(disco-param get $(hostname)/classes/$(disco-param get $(hostname)/current_module)/some/module/specific/path)
|
||||
|
||||
How to deploy stuff
|
||||
=====
|
||||
|
||||
DISCO uses rsync(+ssh) to distribute files, and bash to execute supporting scripts. It has a
|
||||
rudimentary dependency mechanism implemented via a topological sort.
|
||||
|
||||
Essentially, to deploy something, you need 3 things:
|
||||
|
||||
- some static files on an rsync server
|
||||
- some bash templates (scripts that output what their content should be) on an rsync server
|
||||
- some scripts that may or may not do something with those files and templates
|
||||
|
||||
Scripts
|
||||
=====
|
||||
|
||||
DISCO uses bash for a scripting and templating engine. Instead of writing a custom DSL that lets
|
||||
you specify operations (like Puppet did) or utilize a higher level language (like Chef did with
|
||||
ruby), DISCO just uses the proven bash shell.
|
||||
|
||||
Every time your module is executed (e.g. every time the disco client executes), all of the scripts
|
||||
are executed. The order of execution is determined by alphabetically sorting the filenames, rc.d
|
||||
style; so naming your scripts as 00-fix_perms.sh, 10-correct_nodes.sh, etc, will cause them to be
|
||||
executed in the proper order. This prevents you from having to create a separate file that describes
|
||||
the execution order.
|
||||
|
||||
Files vs Templates
|
||||
=====
|
||||
|
||||
Files and Templates are delivered exactly the same way - via rsync.
|
||||
|
||||
Files are static files who are delivered on to the disk, and no more operations are done to them.
|
||||
They are delivered with the same permissions that they are given by the rsync repository.
|
||||
|
||||
Templates are bash scripts who are delivered on to the disk, and then they are executed, with their
|
||||
file contents replaced by their output. Templates are subject to all the same restrictions as scripts
|
||||
(be mindful of the constraints of $NOOP), and in addition, they are ALWAYS interpolated in the safe
|
||||
NOOP execution environment (file modifications will be discarded, and only rudimentary bash builtins
|
||||
are enabled). Templates have access to all client parameters via the disco-param command. Templates
|
||||
will end up with the same permissions that rsync gives them.
|
||||
|
||||
Module Layout
|
||||
=====
|
||||
|
||||
A disco module (also called a "disco ball" for fun) looks like this:
|
||||
|
||||
MODULE
|
||||
___ requires
|
||||
___ parameters
|
||||
├── files/
|
||||
├── scripts/
|
||||
└── templates/
|
||||
|
||||
Your module can theoretically pull files, scripts, and templates from any location that can be
|
||||
reached via rsync; however, it is generally considerd good form to include all things relevant
|
||||
to your module, inside its disco ball. The disco ball is then placed in an accessible location
|
||||
on the rsync server, and the disco client will pull all modules, files, scripts, and templates
|
||||
relevant to its execution, and run them.
|
||||
|
||||
ALL MODULE FILES, SCRIPTS, AND TEMPLATES ARE DELIVERED RELATIVE TO / ON THE CLIENT.
|
||||
|
||||
MODULE/requires
|
||||
=====
|
||||
|
||||
This file lists, one name per line, the names of other modules that must be installed on this
|
||||
node in order for this module to install correctly. This is used to create a dependency graph,
|
||||
and thereby determine execution order.
|
||||
|
||||
This file is optional.
|
||||
|
||||
MODULE/parameters
|
||||
=====
|
||||
|
||||
Each module can define default parameters which will be made available to all clients using the module.
|
||||
These parameters will be merged together on the client at module fetch time, and any node-specific
|
||||
parameters will override any default parameters specified here (they are rsync'ed over the top of each
|
||||
other). These parameters will be rooted at /MODULE_NAME/... .
|
||||
|
||||
This tree is optional.
|
||||
DISCO is in the very early alpha stages of development. At this point, I am fairly confident
|
||||
that it won't destroy the system you run it on, but I still only run it on disposable virtual
|
||||
machines until it reaches a higher level of maturity.
|
||||
|
||||
Server Side Setup
|
||||
=====
|
||||
|
||||
Reference in New Issue
Block a user