Libipc presentation update.
parent
eb99ba6cb2
commit
6fa8f31dd4
208
docs/libipc.md
208
docs/libipc.md
|
@ -36,18 +36,17 @@ Infrastructure
|
||||||
## What solution?
|
## What solution?
|
||||||
|
|
||||||
MAKE APPLICATIONS, NOT LIBRARIES
|
MAKE APPLICATIONS, NOT LIBRARIES
|
||||||
* apps talking to apps
|
* apps talking to apps
|
||||||
|
|
||||||
Create an abstraction for
|
Create an abstraction for libraries
|
||||||
|
|
||||||
libraries
|
|
||||||
* languages, implementations, code modifications in general
|
* languages, implementations, code modifications in general
|
||||||
|
|
||||||
network code
|
Create an abstraction for network code
|
||||||
* networking: performed by dedicated services
|
|
||||||
- example: TCPd, UDPd, TLSd, HTTPd...
|
|
||||||
- LibIPC is independant from protocols and formats
|
|
||||||
* applications think communications are local
|
* applications think communications are local
|
||||||
|
* networking is performed by dedicated services
|
||||||
|
* examples: TCPd, UDPd, TLSd, HTTPd...
|
||||||
|
* apps are independant from protocols and formats
|
||||||
|
(unless they are fundamentaly network-related)
|
||||||
|
|
||||||
## In practice
|
## In practice
|
||||||
|
|
||||||
|
@ -154,7 +153,7 @@ Wait on file descriptors with poll(2)
|
||||||
- slow, but available everywhere
|
- slow, but available everywhere
|
||||||
- may upgrade to libevent
|
- may upgrade to libevent
|
||||||
|
|
||||||
## LibIPC history (1/2)
|
## LibIPC history (1/3)
|
||||||
|
|
||||||
1. based on pipes
|
1. based on pipes
|
||||||
* because we gotta go fast!
|
* because we gotta go fast!
|
||||||
|
@ -168,7 +167,7 @@ Wait on file descriptors with poll(2)
|
||||||
#pause
|
#pause
|
||||||
* ... wait, does select(2) support more than 1024 connections?
|
* ... wait, does select(2) support more than 1024 connections?
|
||||||
|
|
||||||
## LibIPC history (2/2)
|
## LibIPC history (2/3)
|
||||||
|
|
||||||
3. rewrite using poll(2)
|
3. rewrite using poll(2)
|
||||||
* many bugfixes later, way more tested than before
|
* many bugfixes later, way more tested than before
|
||||||
|
@ -177,30 +176,106 @@ Wait on file descriptors with poll(2)
|
||||||
|
|
||||||
Still wasn't as simple as I wanted
|
Still wasn't as simple as I wanted
|
||||||
|
|
||||||
## Current implementation of libIPC
|
## LibIPC history (3/3)
|
||||||
|
|
||||||
Written in Zig
|
4. rewrite in Zig
|
||||||
|
* still uses poll(2) (at least for now)
|
||||||
|
* C-compatible bindings are available
|
||||||
|
|
||||||
|
## Why Zig? (1/2)
|
||||||
|
|
||||||
|
error management is built-in and mandatory
|
||||||
|
|
||||||
|
simpler to read and write
|
||||||
|
* nicer data structures (contain functions)
|
||||||
|
* less code redundancy (defer, more generic functions)
|
||||||
|
* no more C's pitfalls
|
||||||
|
* fully qualified names
|
||||||
|
|
||||||
|
## Why Zig? (2/2)
|
||||||
|
|
||||||
|
better standard library
|
||||||
|
* usual structures: lists, hashtables
|
||||||
|
* log system
|
||||||
|
|
||||||
|
memory management is simpler, more secure and more flexible
|
||||||
|
|
||||||
|
better at exposing bugs (better type system)
|
||||||
|
|
||||||
|
simpler to cross-compile: same standard library for all OSs
|
||||||
|
|
||||||
|
## Current implementation of libIPC
|
||||||
|
|
||||||
bindings available in Crystal
|
bindings available in Crystal
|
||||||
* as well as fancy mappings: JSON and CBOR class serialization
|
* as well as fancy mappings: JSON and CBOR class serialization
|
||||||
|
|
||||||
#pause
|
#pause
|
||||||
epoll (Linux) and kqueue (*BSD) were avoided
|
epoll (Linux) and kqueue (*BSD) were avoided
|
||||||
* 'cause callbacks hell => harder to read and to write code
|
* because callbacks hell => harder to read and to write code
|
||||||
#pause
|
#pause
|
||||||
* but we need them for better performances with many connections
|
* still a possibility for someday, not the priority right now
|
||||||
though, API should stay the same for non threaded applications (simple implementation)
|
|
||||||
|
|
||||||
#pause
|
#pause
|
||||||
LibIPC doesn't handle parallelism, yet
|
LibIPC doesn't handle parallelism, yet
|
||||||
|
|
||||||
## How libIPC works
|
## How libIPC works (in Zig)
|
||||||
|
|
||||||
LibIPC has a high level API for the user
|
LibIPC has a high level API
|
||||||
|
|
||||||
|
var context = try Context.init(allocator);
|
||||||
|
defer context.deinit();
|
||||||
|
|
||||||
|
#pause
|
||||||
|
var pong_fd = try context.connect_service ("pong");
|
||||||
|
var message = try Message.init (pong_fd, allocator, "hello");
|
||||||
|
try context.schedule (message);
|
||||||
|
|
||||||
|
## How libIPC works (in Zig)
|
||||||
|
|
||||||
|
var event = try context.wait_event();
|
||||||
|
|
||||||
|
switch (event.t) {
|
||||||
|
...
|
||||||
|
}
|
||||||
|
|
||||||
|
## How libIPC works (in Zig)
|
||||||
|
|
||||||
|
var event = try context.wait_event();
|
||||||
|
|
||||||
|
switch (event.t) {
|
||||||
|
|
||||||
|
.CONNECTION => {
|
||||||
|
print ("New client!\n", .{});
|
||||||
|
},
|
||||||
|
|
||||||
|
...
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
## How libIPC works (in Zig)
|
||||||
|
|
||||||
|
var event = try context.wait_event();
|
||||||
|
|
||||||
|
switch (event.t) {
|
||||||
|
|
||||||
|
.CONNECTION => {
|
||||||
|
print ("New client!\n", .{});
|
||||||
|
},
|
||||||
|
|
||||||
|
.MESSAGE_RX => {
|
||||||
|
if (event.m) |m| {
|
||||||
|
print ("a message has been received: {s}\n", .{m});
|
||||||
|
}
|
||||||
|
}
|
||||||
|
...
|
||||||
|
}
|
||||||
|
|
||||||
|
## How libIPC works (bindings)
|
||||||
|
|
||||||
1. init a connection (client) or create an unix socket (service)
|
1. init a connection (client) or create an unix socket (service)
|
||||||
|
|
||||||
example: ipc_service_init (context, "service")
|
ipc_connect_service (context, &fd, service_name, service_len)
|
||||||
|
ipc_service_init (context, &fd, service_name, service_len)
|
||||||
|
|
||||||
#pause
|
#pause
|
||||||
2. loop, wait for events
|
2. loop, wait for events
|
||||||
|
@ -209,14 +284,11 @@ LibIPC has a high level API for the user
|
||||||
example:
|
example:
|
||||||
|
|
||||||
while(1) {
|
while(1) {
|
||||||
wait_event (context, &event, &timer)
|
ipc_wait_event (context, &type, &index, &fd, buffer, &buffer_len)
|
||||||
switch (event.type) {
|
switch (type) {
|
||||||
case IPC_CONNECTION : ...
|
case IPC_CONNECTION : ...
|
||||||
case IPC_DISCONNECTION : ...
|
case IPC_DISCONNECTION : ...
|
||||||
case IPC_MESSAGE: {
|
case IPC_MESSAGE: ...
|
||||||
struct ipc_message *m = event.m;
|
|
||||||
...
|
|
||||||
}
|
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
@ -225,20 +297,15 @@ LibIPC has a high level API for the user
|
||||||
3. send messages
|
3. send messages
|
||||||
|
|
||||||
```c
|
```c
|
||||||
struct ipc_message m
|
ipc_schedule (context, fd, buffer, buffer_len)
|
||||||
m.payload = ...
|
or
|
||||||
m.length = strlen(m.payload)
|
ipc_write (context, fd, buffer, buffer_len)
|
||||||
m.fd = event.fd
|
|
||||||
m.type = ...
|
|
||||||
m.user_type = ...
|
|
||||||
ipc_write (context, &m)
|
|
||||||
```
|
```
|
||||||
|
|
||||||
#pause
|
#pause
|
||||||
4. add and remove fd from the context
|
4. add a file descriptor to listen to
|
||||||
|
|
||||||
ipc_add_fd (context, fd)
|
ipc_add_external (context, fd)
|
||||||
ipc_del_fd (context, fd)
|
|
||||||
|
|
||||||
## How libIPC works
|
## How libIPC works
|
||||||
|
|
||||||
|
@ -259,70 +326,51 @@ Example: websocketd.
|
||||||
|
|
||||||
ipc_switching_callbacks (context, client_fd, cb_in, cb_out)
|
ipc_switching_callbacks (context, client_fd, cb_in, cb_out)
|
||||||
|
|
||||||
## libIPC internal structures
|
## libIPC internal structures (1/2)
|
||||||
|
|
||||||
Main goal: simplest possible structures
|
Main goal: simplest possible structures
|
||||||
|
|
||||||
Examples:
|
Examples (nothing hidden):
|
||||||
|
|
||||||
Message
|
Message {
|
||||||
|
fd: i32 => File descriptor concerned about this message.
|
||||||
struct ipc_message {
|
payload: []u8 => Actual payload.
|
||||||
char type; => Internal message type, used by protocol daemons.
|
allocator: std.mem.Allocator => Memory management.
|
||||||
char user_type; => User-defined message type (arbitrary).
|
|
||||||
int fd; => File descriptor concerned about this message.
|
|
||||||
uint32_t length; => Payload length.
|
|
||||||
char *payload; => Actual payload.
|
|
||||||
};
|
};
|
||||||
|
|
||||||
Context of the whole networking state
|
Event {
|
||||||
|
t: Event.Type => Example: connection, message tx, ...
|
||||||
struct ipc_ctx {
|
m: ?Message => Message, if there is one.
|
||||||
struct ipc_connection_info *cinfos; => Keeps track of connections.
|
index: usize => (Internal stuff).
|
||||||
struct pollfd *pollfd; => List of "pollfd" structures within cinfos,
|
originfd: i32 => File descriptor related to the event.
|
||||||
so we can pass it to poll(2).
|
|
||||||
size_t size; => Size of the connection list.
|
|
||||||
struct ipc_messages tx; => Messages to send.
|
|
||||||
struct ipc_switchings switchdb; => Relations between fd.
|
|
||||||
};
|
};
|
||||||
|
|
||||||
|
## libIPC internal structures (2/2)
|
||||||
|
|
||||||
|
Context structure is slightly more complicated, but _reasonable_.
|
||||||
|
|
||||||
|
Context {
|
||||||
|
rundir: [] u8, // Where the UNIX sockets are.
|
||||||
|
pollfd: PollFD, // File descriptors to manage.
|
||||||
|
tx: Messages, // Messages to send, once their fd is available.
|
||||||
|
...
|
||||||
|
};
|
||||||
|
|
||||||
|
The rest is implementation details (and more advanced usage of LibIPC).
|
||||||
|
|
||||||
## Future of libIPC
|
## Future of libIPC
|
||||||
|
|
||||||
LibIPC will be rewritten in Zig
|
|
||||||
|
|
||||||
* simpler to read and write
|
|
||||||
* data structures are simpler to create with comptime
|
|
||||||
* less code redundancy (more generic functions)
|
|
||||||
* already existing error management as written in current libIPC
|
|
||||||
* already existing loggin system
|
|
||||||
* no more C's pitfalls
|
|
||||||
|
|
||||||
* way better at exposing bugs
|
|
||||||
* thanks to a better type system
|
|
||||||
|
|
||||||
* way safer: cannot ignore errors
|
|
||||||
* so I won't
|
|
||||||
|
|
||||||
* simpler to cross-compile: same standard library for every OSs
|
|
||||||
* simpler (and more secure) memory management
|
|
||||||
|
|
||||||
|
|
||||||
Also: this won't change existing bindings! No excuses!
|
|
||||||
|
|
||||||
|
|
||||||
## Why not use it?
|
## Why not use it?
|
||||||
|
|
||||||
Current limitations
|
Current limitations
|
||||||
|
|
||||||
* performances (libIPC is based on poll(2), not epoll nor kqueue)
|
* performances (libIPC is based on poll(2), not epoll nor kqueue)
|
||||||
* it really isn't an issue until you have hundreds or thousands of clients
|
* it really isn't an issue until you have hundreds of clients
|
||||||
|
* LibIPC could someday use libevent
|
||||||
|
|
||||||
* parallelism is not permitted
|
* nothing in libIPC is thread-safe
|
||||||
nothing in libIPC is thread-safe
|
|
||||||
|
|
||||||
|
These limitations are the price for a simple implementation.
|
||||||
The future Zig implementation will overcome these issues.
|
|
||||||
|
|
||||||
## Questions?
|
## Questions?
|
||||||
|
|
||||||
|
|
Reference in New Issue