Push tcp #2
Merged
Push tcp #2
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
We have a small leak here, that clp sockets keep referenced after the peer closed. Fix.
The design of Donald::Select makes use of Donald::Callback objects,
which are objects conainting a sub and call arguments. However,
because of the pattern
sub do_something_later() {
my ($cb_or_sub,@args)=@_;
$store_callback_somehere=new Donald::Callback($cb_or_sub,@args);
}
do_something_later(\&callback,$arg1,$arg2)
the caller doesn't have a reference to the Donald::Callback object,
which makes its diffucult to identify it, e.g. to cancel the callback.
We want to change the design to accept only references to subs as
callbacks. Instead of passing arguments, we exepect the caller to make
use of closures to pass data to the callback if needed.
sub do_something_later() {
my ($cb)=@_;
$store_callback_somewhere=$cb;
}
do_something_later(sub{callback($arg1,$args)});
Instead of changing the API of Donald::Select, we import the code
directly into clusterd to make the modifications here.
Use the namespace My:: for packages declared inside the main script.
We imported some lines which do no longer have a function now. Remove them.
Non-functional change to bring source into sync with another working branch.
Use refenerces to subs as callback arguments. If the caller wants to pass additional arguments, he can use closures.
This hash is not used but it prevents these sockets to be able to be closed by running out of scope.
The command "clusterd push file..." has a new syntax with a command verb. (as opposed to "clusterd --push file") The push command is distributed via the area routers to the cluster daemons on all nodes. It is intended that the cluster daemons call back over tcp to the originator to pull the file if needed.
When a daemon receices a push command, it checks whether it already has the offered file or not. If not, it calls back to the daemon where the push originated and pulls the file over tcp.
Implement the server side to pull a file over tcp.
While 5 seconds seems to be enough for normal behaviour, we might need more time if the daemon is slowed down .e.g. by strace.
With 234 hosts pulling a file we failed to service a single on with the old setting listen=1.
Sign in
to join this conversation on GitHub.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
New command
can be used to distribute files via tcp. Because unlike
clusterd --push filewe do not use udp broadcast for the file content, we don't have to limit the file size.