Obsolete
/
libipc-old
Archived
3
0
Fork 0

Libipc presentation update.

master
Philippe Pittoli 2023-01-21 01:47:58 +01:00
parent eb99ba6cb2
commit 6fa8f31dd4
1 changed files with 128 additions and 80 deletions

View File

@ -36,18 +36,17 @@ Infrastructure
## What solution? ## What solution?
MAKE APPLICATIONS, NOT LIBRARIES MAKE APPLICATIONS, NOT LIBRARIES
* apps talking to apps * apps talking to apps
Create an abstraction for Create an abstraction for libraries
libraries
* languages, implementations, code modifications in general * languages, implementations, code modifications in general
network code Create an abstraction for network code
* networking: performed by dedicated services
- example: TCPd, UDPd, TLSd, HTTPd...
- LibIPC is independant from protocols and formats
* applications think communications are local * applications think communications are local
* networking is performed by dedicated services
* examples: TCPd, UDPd, TLSd, HTTPd...
* apps are independant from protocols and formats
(unless they are fundamentaly network-related)
## In practice ## In practice
@ -154,7 +153,7 @@ Wait on file descriptors with poll(2)
- slow, but available everywhere - slow, but available everywhere
- may upgrade to libevent - may upgrade to libevent
## LibIPC history (1/2) ## LibIPC history (1/3)
1. based on pipes 1. based on pipes
* because we gotta go fast! * because we gotta go fast!
@ -168,7 +167,7 @@ Wait on file descriptors with poll(2)
#pause #pause
* ... wait, does select(2) support more than 1024 connections? * ... wait, does select(2) support more than 1024 connections?
## LibIPC history (2/2) ## LibIPC history (2/3)
3. rewrite using poll(2) 3. rewrite using poll(2)
* many bugfixes later, way more tested than before * many bugfixes later, way more tested than before
@ -177,30 +176,106 @@ Wait on file descriptors with poll(2)
Still wasn't as simple as I wanted Still wasn't as simple as I wanted
## Current implementation of libIPC ## LibIPC history (3/3)
Written in Zig 4. rewrite in Zig
* still uses poll(2) (at least for now)
* C-compatible bindings are available
## Why Zig? (1/2)
error management is built-in and mandatory
simpler to read and write
* nicer data structures (contain functions)
* less code redundancy (defer, more generic functions)
* no more C's pitfalls
* fully qualified names
## Why Zig? (2/2)
better standard library
* usual structures: lists, hashtables
* log system
memory management is simpler, more secure and more flexible
better at exposing bugs (better type system)
simpler to cross-compile: same standard library for all OSs
## Current implementation of libIPC
bindings available in Crystal bindings available in Crystal
* as well as fancy mappings: JSON and CBOR class serialization * as well as fancy mappings: JSON and CBOR class serialization
#pause #pause
epoll (Linux) and kqueue (*BSD) were avoided epoll (Linux) and kqueue (*BSD) were avoided
* 'cause callbacks hell => harder to read and to write code * because callbacks hell => harder to read and to write code
#pause #pause
* but we need them for better performances with many connections * still a possibility for someday, not the priority right now
though, API should stay the same for non threaded applications (simple implementation)
#pause #pause
LibIPC doesn't handle parallelism, yet LibIPC doesn't handle parallelism, yet
## How libIPC works ## How libIPC works (in Zig)
LibIPC has a high level API for the user LibIPC has a high level API
var context = try Context.init(allocator);
defer context.deinit();
#pause
var pong_fd = try context.connect_service ("pong");
var message = try Message.init (pong_fd, allocator, "hello");
try context.schedule (message);
## How libIPC works (in Zig)
var event = try context.wait_event();
switch (event.t) {
...
}
## How libIPC works (in Zig)
var event = try context.wait_event();
switch (event.t) {
.CONNECTION => {
print ("New client!\n", .{});
},
...
}
## How libIPC works (in Zig)
var event = try context.wait_event();
switch (event.t) {
.CONNECTION => {
print ("New client!\n", .{});
},
.MESSAGE_RX => {
if (event.m) |m| {
print ("a message has been received: {s}\n", .{m});
}
}
...
}
## How libIPC works (bindings)
1. init a connection (client) or create an unix socket (service) 1. init a connection (client) or create an unix socket (service)
example: ipc_service_init (context, "service") ipc_connect_service (context, &fd, service_name, service_len)
ipc_service_init (context, &fd, service_name, service_len)
#pause #pause
2. loop, wait for events 2. loop, wait for events
@ -209,14 +284,11 @@ LibIPC has a high level API for the user
example: example:
while(1) { while(1) {
wait_event (context, &event, &timer) ipc_wait_event (context, &type, &index, &fd, buffer, &buffer_len)
switch (event.type) { switch (type) {
case IPC_CONNECTION : ... case IPC_CONNECTION : ...
case IPC_DISCONNECTION : ... case IPC_DISCONNECTION : ...
case IPC_MESSAGE: { case IPC_MESSAGE: ...
struct ipc_message *m = event.m;
...
}
} }
} }
@ -225,20 +297,15 @@ LibIPC has a high level API for the user
3. send messages 3. send messages
```c ```c
struct ipc_message m ipc_schedule (context, fd, buffer, buffer_len)
m.payload = ... or
m.length = strlen(m.payload) ipc_write (context, fd, buffer, buffer_len)
m.fd = event.fd
m.type = ...
m.user_type = ...
ipc_write (context, &m)
``` ```
#pause #pause
4. add and remove fd from the context 4. add a file descriptor to listen to
ipc_add_fd (context, fd) ipc_add_external (context, fd)
ipc_del_fd (context, fd)
## How libIPC works ## How libIPC works
@ -259,70 +326,51 @@ Example: websocketd.
ipc_switching_callbacks (context, client_fd, cb_in, cb_out) ipc_switching_callbacks (context, client_fd, cb_in, cb_out)
## libIPC internal structures ## libIPC internal structures (1/2)
Main goal: simplest possible structures Main goal: simplest possible structures
Examples: Examples (nothing hidden):
Message Message {
fd: i32 => File descriptor concerned about this message.
struct ipc_message { payload: []u8 => Actual payload.
char type; => Internal message type, used by protocol daemons. allocator: std.mem.Allocator => Memory management.
char user_type; => User-defined message type (arbitrary).
int fd; => File descriptor concerned about this message.
uint32_t length; => Payload length.
char *payload; => Actual payload.
}; };
Context of the whole networking state Event {
t: Event.Type => Example: connection, message tx, ...
struct ipc_ctx { m: ?Message => Message, if there is one.
struct ipc_connection_info *cinfos; => Keeps track of connections. index: usize => (Internal stuff).
struct pollfd *pollfd; => List of "pollfd" structures within cinfos, originfd: i32 => File descriptor related to the event.
so we can pass it to poll(2).
size_t size; => Size of the connection list.
struct ipc_messages tx; => Messages to send.
struct ipc_switchings switchdb; => Relations between fd.
}; };
## libIPC internal structures (2/2)
Context structure is slightly more complicated, but _reasonable_.
Context {
rundir: [] u8, // Where the UNIX sockets are.
pollfd: PollFD, // File descriptors to manage.
tx: Messages, // Messages to send, once their fd is available.
...
};
The rest is implementation details (and more advanced usage of LibIPC).
## Future of libIPC ## Future of libIPC
LibIPC will be rewritten in Zig
* simpler to read and write
* data structures are simpler to create with comptime
* less code redundancy (more generic functions)
* already existing error management as written in current libIPC
* already existing loggin system
* no more C's pitfalls
* way better at exposing bugs
* thanks to a better type system
* way safer: cannot ignore errors
* so I won't
* simpler to cross-compile: same standard library for every OSs
* simpler (and more secure) memory management
Also: this won't change existing bindings! No excuses!
## Why not use it? ## Why not use it?
Current limitations Current limitations
* performances (libIPC is based on poll(2), not epoll nor kqueue) * performances (libIPC is based on poll(2), not epoll nor kqueue)
* it really isn't an issue until you have hundreds or thousands of clients * it really isn't an issue until you have hundreds of clients
* LibIPC could someday use libevent
* parallelism is not permitted * nothing in libIPC is thread-safe
nothing in libIPC is thread-safe
These limitations are the price for a simple implementation.
The future Zig implementation will overcome these issues.
## Questions? ## Questions?