native seems to be broken and delivers newlines into the logile

unter die knute des bee

we now create release tags
.e.g. bee init https://github.molgen.mpg.de/donald/clusterd/archive/v1.104.tar.gz would work

for the rolling upgrade we need to request another rpc version, because
the current receiver trusts the sender to send only plain files

symlinks need to be send to a new rpc "filedata.2" because the old rpc
can't handle it. the updated receiver is installed under the old name
"filedata" too in case older daemons send push

* symlinks:
  update receiver to handle symlinks
  enable push to send symlink
  add lchown() , lmitime()

This command might be usefull, e.g. if sshd is not responding.

Examples:

  clusterd --exec @theinternet restart sshd.service
  clusterd --exec @theinternet dmesg
  clusterd --exec @theinternet 'grep sshd /var/log/messages|tail -100'

The remote command is executed by bash, so the pipe in the last example
is on the remote node

stdout and stderr are delivered seperatly and might be redirected to
different channels on the local side.

If the remote command exits with a non-zero exit status, the local
command fails with exit status 1.

The slave part already existed, because we once had a remote
execution command. We removed it, because it was considered to
be to dangerous. The former remote execution command allowed parallel
execution on all nodes, which enabled an admin to kill all systems
by mistake in an instance. Now we only enable remote execution on a
single node.

* add-remote-exec:
  Implement --exec @node cmd [args...]

We have a small leak here, that clp sockets keep referenced after the
peer closed. Fix.

The design of Donald::Select makes use of Donald::Callback objects,
which are objects conainting a sub and call arguments. However,
because of the pattern

    sub do_something_later() {
        my ($cb_or_sub,@args)=@_;
            $store_callback_somehere=new Donald::Callback($cb_or_sub,@args);
        }

    do_something_later(\&callback,$arg1,$arg2)

the caller doesn't have a reference to the Donald::Callback object,
which makes its diffucult to identify it, e.g. to cancel the callback.

We want to change the design to accept only references to subs as
callbacks. Instead of passing arguments, we exepect the caller to make
use of closures to pass data to the callback if needed.

    sub do_something_later() {
        my ($cb)=@_;
        $store_callback_somewhere=$cb;
    }

    do_something_later(sub{callback($arg1,$args)});

Instead of changing the API of Donald::Select, we import the code
directly into clusterd to make the modifications here.

Use the namespace My:: for packages declared inside the main script.

We imported some lines which do no longer have a function now. Remove
them.

Non-functional change to bring source into sync with another working
branch.

Use refenerces to subs as callback arguments. If the caller wants to
pass additional arguments, he can use closures.

This hash is not used but it prevents these sockets to be able to
be closed by running out of scope.

The command "clusterd push file..." has a new syntax with a command
verb. (as opposed to "clusterd --push file")

The push command is distributed via the area routers to the cluster
daemons on all nodes. It is intended that the cluster daemons call
back over tcp to the originator to pull the file if needed.

When a daemon receices a push command, it checks whether it already has
the offered file or not. If not, it calls back to the daemon where the
push originated and pulls the file over tcp.

Implement the server side to pull a file over tcp.

While 5 seconds seems to be enough for normal behaviour, we might need
more time if the daemon is slowed down .e.g. by strace.

With 234 hosts pulling a file we failed to service a single on with the
old setting listen=1.

Push tcp

If we have dead nfs mounts, lsof might hangover. Limit a timeout and
return an error message if lsof fails.

Refactor the code, so that the mltime specific settings are done from
inside the machine type switch. This makes it easier to add another
machine type which can set the time of a symlink.

amd automounted is gone on all systems, so don't try to call amq.

Currently we may move received files in place, even whe the write or
close were unsuccessfull (eg. disk full). This is a serious bug and we
may end up with an empty passwd.

Add checks for all write() and close() calls.
Also log rename errors.

The clusterd repository has been prepared by

    git filter-branch \
        --msg-filter 'sed -r "1s/^(clusterd: )?/clusterd: /"' \
        --index-filter 'git ls-files -s | sed "s-\t\"*-&clusterd/-" | GIT_INDEX_FILE=$GIT_INDEX_FILE.new git update-index --index-info && mv "$GIT_INDEX_FILE.new" "$GIT_INDEX_FILE"' \
        -- master

to move all files into a subdirectory clusterd/ and make sure,
the commit messages are prefixed wih "clusterd:".

Not merge its master branch.

* clusterd/master:
  clusterd: Remove redundant blank lines.
  clusterd: Add checks for write errors
  clusterd: Remove call to amq
  clusterd: Add ppc64le architecture
  clusterd: Refactor lmtime setting
  clusterd: Increase lsof timeout
  clusterd: Do not resolve hostname when doing lsof
  clusterd: Bump up version
  clusterd: Add timeout for lsof
  clusterd: Bump up version
  clusterd: Increase listen queue size for clp port
  clusterd: Increase TCP timeout from 5 to 30
  clusterd: Implement PULL in daemon
  clusterd: Implement PUSH in daemon
  clusterd: Add command "clusterd push files..."
  clusterd: Add send_tcp_cp() to send a cluster protocol message over tcp
  clusterd: Add functions for nonblocking tcp to My::Select::INET
  clusterd: Remove clp_sockets hash
  clusterd: Remove/Move imports
  clusterd: Remove My::Callback
  clusterd: Remove/fix some comments
  clusterd: Remove unused sub heartbeat.
  clusterd: Remove declarations no longer needeid after import
  clusterd: Rename Donald::Select to My::Select
  clusterd: Import Donald::Select into script
  clusterd: Remove mlx ping
  clusterd: Dereference CLP tcp socket after hangup
  clusterd: Clean up indentation mix
  clusterd: implement --make-automaps
  clusterd: bump up version
  clusterd: remove some debug relicts
  clusterd: Implement --exec @node cmd [args...]
  clusterd: fix shebang to use /usr/bin/perl
  clusterd: bump up revision
  clusterd: update receiver to handle symlinks
  clusterd: enable push to send symlink
  clusterd: add lchown() , lmitime()
  clusterd: enable push to send empty files
  clusterd: increase push file limit from 5*8*1024 to 8*8*1024
  clusterd: make push_file errors like "to big" fatal
  clusterd: change wait to happen before broadcast if we need to throttle
  clusterd: style: remove some extra whitespace
  clusterd: update comment pointing to github home
  clusterd: bump up revision
  clusterd: ping mlx once a day
  clusterd: remove mkbee.sh
  clusterd: nach /usr/sbin installieren aber in /sbin aufrufen is nix gut
  clusterd: unter die knute des bee
  clusterd: use unix socket for syslog
  clusterd: we live on github now
  clusterd: add clusterd.service
  clusterd: add --flush-gidcache
  clusterd: add primitive Makefile
  clusterd: initial

Add clusterd files to installation script and remove its obsolete
stand-alone Makefile.