Obsolete
/
ipcd
Archived
3
0
Fork 0

Networkd => Plumberd, websocketd working.

master
Philippe PITTOLI 2019-11-04 08:56:57 +01:00
parent 550f4e77ae
commit 176d8d1507
13 changed files with 404 additions and 77 deletions

View File

@ -1,15 +1,15 @@
Networkd is a program to handle networking for all other software. Plumberd is a program to handle networking for all other software.
# WARNING # WARNING
Security is TBD. Currently, only TCPd is implemented, which means no communication security. Security is TBD. Currently, only TCPd is implemented, which means no communication security.
# Networkd functionalities # Plumberd functionalities
## firewall ## firewall
`Networkd` has to filter the connections to local services. `Plumberd` has to filter the connections to local services.
```Warning ```Warning
WIP. WIP.
@ -17,7 +17,7 @@ WIP.
## authentication ## authentication
`Networkd` has to authenticate clients asking for a service. `Plumberd` has to authenticate clients asking for a service.
```Warning ```Warning
WIP. WIP.
@ -49,31 +49,31 @@ This program can be used as follow:
```sh ```sh
# with some static rules # with some static rules
networkd --allow in authd tls:example.com --deny in * * --allow out pong tls:pong.example.com:9000 plumberd --allow in authd tls:example.com --deny in * * --allow out pong tls:pong.example.com:9000
networkd --redirect authd nextversion-authd plumberd --redirect authd nextversion-authd
``` ```
## usage examples ## usage examples
`networkd` is requested each time a client is launched when the right environment variable is used. `plumberd` is requested each time a client is launched when the right environment variable is used.
For example, we want to connect to a distant `authd` service: For example, we want to connect to a distant `authd` service:
IPC_NETWORKD="authd tls://user@passwd:example.com:9000/authd" IPC_NETWORKD="authd tls://user@passwd:example.com:9000/authd"
```Warning ```Warning
Currently, the networkd only works with tcp and unix routes. Currently, the plumberd only works with tcp and unix routes.
``` ```
IPC_NETWORKD="pongd tcp://example.com:9000/pongd" IPC_NETWORKD="pongd tcp://example.com:9000/pongd"
# Changelog # Changelog
* v0.1: (current) networkd (redirections), tcpd * v0.1: (current) plumberd (redirections), tcpd
* `networkd` understands URIs (`tcp://example.com/service` or `unix:///service`) * `plumberd` understands URIs (`tcp://example.com/service` or `unix:///service`)
* `tcp` scheme is understood: `networkd` contacts the `tcpd` service * `tcp` scheme is understood: `plumberd` contacts the `tcpd` service
* `unix` scheme is understood: `networkd` performs a redirection * `unix` scheme is understood: `plumberd` performs a redirection
# Roadmap # Roadmap
@ -87,16 +87,16 @@ Currently, the networkd only works with tcp and unix routes.
* v1.0: TBD * v1.0: TBD
# Networkd explanations # Plumberd explanations
1. client contacts `networkd` 1. client contacts `plumberd`
1. `networkd` understand the request from the client then contacts the local service responsible for the communication protocol required 1. `plumberd` understand the request from the client then contacts the local service responsible for the communication protocol required
1. once the distant connection is established (between the two `tlsd` services for example) `networkd` provides a file descriptor to the client 1. once the distant connection is established (between the two `tlsd` services for example) `plumberd` provides a file descriptor to the client
1. finally, the client can perform requests to the distant service transparently 1. finally, the client can perform requests to the distant service transparently
during the connection: during the connection:
client <-> networkd <-> tlsd <=> tlsd <-> networkd <-> service client <-> plumberd <-> tlsd <=> tlsd <-> plumberd <-> service
then: then:

View File

@ -1,11 +1,11 @@
name: networkd name: plumberd
version: 0.1.1 version: 0.2.0
authors: authors:
- karchnu <karchnu@karchnu.fr> - karchnu <karchnu@karchnu.fr>
description: | description: |
Networkd allows IPC clients to contact remote services. Plumberd allows IPC clients to contact remote services.
dependencies: dependencies:
ipc: ipc:
@ -18,9 +18,12 @@ targets:
pongd: pongd:
main: src/pongd.cr main: src/pongd.cr
networkd: plumberd:
main: src/main.cr main: src/main.cr
admind:
main: src/admind.cr
tcpd: tcpd:
main: src/tcpd.cr main: src/tcpd.cr
tcp: tcp:

92
src/admind.cr Normal file
View File

@ -0,0 +1,92 @@
require "option_parser"
require "ipc"
require "./colors"
require "json"
verbosity = 1
service_name = "admind"
OptionParser.parse! do |parser|
parser.on "-s service_name", "--service-name service_name", "URI" do |optsn|
service_name = optsn
end
parser.on "-v verbosity", "--verbosity verbosity", "Verbosity (0 = nothing is printed, 1 = only events, 2 = events and messages). Default: 1" do |optsn|
verbosity = optsn.to_i
end
parser.on "-h", "--help", "Show this help" do
puts parser
exit 0
end
end
class Rectangle
JSON.mapping(
x: UInt32,
y: UInt32
)
def initialize(@x : UInt32, @y : UInt32)
end
end
class WidgetTree
JSON.mapping(
rectangles: Array(Rectangle)
)
def initialize(@rectangles : Array(Rectangle))
end
end
class Page
JSON.mapping(
page: String,
title: String,
tree: WidgetTree
)
def initialize(@page, @title, @tree)
end
end
IPC::Service.new (service_name) do |event|
case event
when IPC::Event::Timer
if verbosity >= 1
puts "#{CORANGE}IPC::Event::Timer#{CRESET}"
end
when IPC::Event::Connection
if verbosity >= 1
puts "#{CBLUE}IPC::Event::Connection#{CRESET}, client: #{event.connection.fd}"
end
when IPC::Event::Disconnection
if verbosity >= 1
puts "#{CBLUE}IPC::Event::Disconnection#{CRESET}, client: #{event.connection.fd}"
end
when IPC::Event::Message
if verbosity >= 1
puts "#{CGREEN}IPC::Event::Message#{CRESET}, client: #{event.connection.fd}"
if verbosity == 2
puts "#{CBLUE}message: #{event.message} #{CRESET}"
end
end
# JSON message
rectangles = Array(Rectangle).new
rectangles << Rectangle.new 1, 2
rectangles << Rectangle.new 2, 1
rectangles << Rectangle.new 3, 1
widgets = WidgetTree.new(rectangles)
page = Page.new "admin", "This is the Admin Page", widgets
event.connection.send 2, page.to_json
else
if verbosity >= 1
puts "#{CRED}Exception: message = #{event.message} #{CRESET}"
end
end
end

View File

@ -1,16 +1,16 @@
require "./networkd" require "./plumberd"
networkd = NetworkD.new "network" plumberd = NetworkD.new "plumber"
# --deny <in|out> <service> <url> # --deny <in|out> <service> <url>
# --allow <in|ou> <service> <url> # --allow <in|ou> <service> <url>
# --redirect <service> <service> # --redirect <service> <service>
# --redirect <service-name> <new-service-name> <origin-url> <dest-url> # --redirect <service-name> <new-service-name> <origin-url> <dest-url>
networkd.parse_cli ARGV plumberd.parse_cli ARGV
puts networkd.to_s puts plumberd.to_s
networkd.loop do |event| plumberd.loop do |event|
puts "there is an event!" puts "there is an event!"
case event case event
when IPC::Event::Connection when IPC::Event::Connection
@ -27,7 +27,7 @@ networkd.loop do |event|
puts "\033[32mthere is a message\033[00m" puts "\033[32mthere is a message\033[00m"
puts event.message.to_s puts event.message.to_s
networkd.service_lookup event.message, event.connection plumberd.service_lookup event.message, event.connection
end end
end end

View File

@ -1,15 +1,15 @@
require "./rules" require "./rules"
require "./networkdcliparser" require "./plumberdcliparser"
require "ipc" require "ipc"
require "uri" require "uri"
class IPC::NetworkD < IPC::Service class IPC::NetworkD < IPC::Service
# The client asks to networkd to open a connection to a service # The client asks to plumberd to open a connection to a service
# the service name is used unless there is a redirection that is provided to the client through # the service name is used unless there is a redirection that is provided to the client through
# a IPC_NETWORK environment variable # a IPC_NETWORK environment variable
# This environment variable is sent from the client to networkd, that's what is parsed here # This environment variable is sent from the client to plumberd, that's what is parsed here
# parse_lookup_payload extract the right service URI to use from the IPC_NETWORK content # parse_lookup_payload extract the right service URI to use from the IPC_NETWORK content
# sent by the client in the form: # sent by the client in the form:
@ -80,7 +80,7 @@ class IPC::NetworkD < IPC::Service
# TODO: for remote services, we have to connect to communication service # TODO: for remote services, we have to connect to communication service
# these communication services need an URI to work on # these communication services need an URI to work on
# The protocol: # The protocol:
# networkd sends an URI to the communication service, which responds with a "OK" message # plumberd sends an URI to the communication service, which responds with a "OK" message
if scheme != "unix" if scheme != "unix"
service.send 1.to_u8, "#{requested_service.to_s}\n" service.send 1.to_u8, "#{requested_service.to_s}\n"
response = service.read response = service.read
@ -97,7 +97,7 @@ class IPC::NetworkD < IPC::Service
raise Exception.new "cannot send the file descriptor of the requested service: #{m}" raise Exception.new "cannot send the file descriptor of the requested service: #{m}"
end end
# finally, the service should be closed in networkd # finally, the service should be closed in plumberd
service.close service.close
rescue e rescue e
puts "\033[31mException during the connection to the requested service #{requested_service}: #{e}\033[00m" puts "\033[31mException during the connection to the requested service #{requested_service}: #{e}\033[00m"
@ -118,8 +118,8 @@ class IPC::NetworkD < IPC::Service
def wait_event(server : IPC::Connection?, &block) : Tuple(LibIPC::EventType, IPC::Message, IPC::Connection) def wait_event(server : IPC::Connection?, &block) : Tuple(LibIPC::EventType, IPC::Message, IPC::Connection)
event = LibIPC::Event.new event = LibIPC::Event.new
# TODO: networkd should be able to transfer messages??? # TODO: plumberd should be able to transfer messages???
r = LibIPC.ipc_wait_event self.pointer, @service_info.pointer, pointerof(event) r = LibIPC.ipc_wait_event self.pointer, @service_info.pointer, pointerof(event), pointerof(@timer)
if r != 0 if r != 0
m = String.new LibIPC.ipc_errors_get (r) m = String.new LibIPC.ipc_errors_get (r)
yield IPC::Exception.new "error waiting for a new event: #{m}" yield IPC::Exception.new "error waiting for a new event: #{m}"

View File

@ -2,6 +2,7 @@ require "option_parser"
require "ipc" require "ipc"
require "./colors" require "./colors"
verbosity = 1
service_name = "pong" service_name = "pong"
OptionParser.parse! do |parser| OptionParser.parse! do |parser|
@ -9,6 +10,10 @@ OptionParser.parse! do |parser|
service_name = optsn service_name = optsn
end end
parser.on "-v verbosity", "--verbosity verbosity", "Verbosity (0 = nothing is printed, 1 = only events, 2 = events and messages). Default: 1" do |optsn|
verbosity = optsn.to_i
end
parser.on "-h", "--help", "Show this help" do parser.on "-h", "--help", "Show this help" do
puts parser puts parser
exit 0 exit 0
@ -18,13 +23,29 @@ end
IPC::Service.new (service_name) do |event| IPC::Service.new (service_name) do |event|
case event case event
when IPC::Event::Timer
if verbosity >= 1
puts "#{CORANGE}IPC::Event::Timer#{CRESET}"
end
when IPC::Event::Connection when IPC::Event::Connection
puts "#{CBLUE}IPC::Event::Connection#{CRESET}, client: #{event.connection.fd}" if verbosity >= 1
puts "#{CBLUE}IPC::Event::Connection#{CRESET}, client: #{event.connection.fd}"
end
when IPC::Event::Disconnection when IPC::Event::Disconnection
puts "#{CBLUE}IPC::Event::Disconnection#{CRESET}, client: #{event.connection.fd}" if verbosity >= 1
puts "#{CBLUE}IPC::Event::Disconnection#{CRESET}, client: #{event.connection.fd}"
end
when IPC::Event::Message when IPC::Event::Message
puts "#{CGREEN}IPC::Event::Message#{CRESET}, client: #{event.connection.fd}" if verbosity >= 1
# puts event.message.to_s puts "#{CGREEN}IPC::Event::Message#{CRESET}, client: #{event.connection.fd}"
if verbosity == 2
puts "#{CBLUE}message: #{event.message} #{CRESET}"
end
end
event.connection.send event.message event.connection.send event.message
else
if verbosity >= 1
puts "#{CRED}Exception: message = #{event.message} #{CRESET}"
end
end end
end end

View File

@ -0,0 +1,25 @@
require "./utils.cr"
begin
message = String.build do |str|
i = 0
80.times do
str << "i=#{i}"
i += 1
end
end
rescue e
puts "Error: #{e}"
exit 1
end
begin
bin = to_message 2, message
rescue e
puts "error in to_message: #{e}"
exit 2
end
print_hexa String.new(bin), "message"

View File

@ -1,14 +1,11 @@
require "io/hexdump" require "io/hexdump"
def to_message (user_type : Int, message : String) def to_message (user_type : Int, message : String)
payload = Bytes.new (6 + message.size) payload = Bytes.new (6 + message.to_slice.size)
# true start # true start
payload[0] = 1.to_u8 payload[0] = 1.to_u8
payload[1] = 0.to_u8 IO::ByteFormat::NetworkEndian.encode message.to_slice.size, (payload + 1)
payload[2] = 0.to_u8
payload[3] = 0.to_u8
payload[4] = message.size.to_u8
# second part: user message # second part: user message
payload[5] = user_type.to_u8 payload[5] = user_type.to_u8

View File

@ -55,16 +55,26 @@ end
begin begin
ws = WebSocket.new(URI.parse(uri)) ws = WebSocket.new(URI.parse(uri))
puts "connection done: sending pong"
ws.on_close do |socket| ws.on_close do |socket|
puts "socket is closing" puts "socket is closing"
exit 0 exit 0
end end
read ws if uri.ends_with? ".JSON"
send ws, "pong" json_message = STDIN.gets_to_end
read ws json_message = json_message.chomp
send ws, to_message(2, "coucou") puts "json_message: #{json_message}"
read ws final_json_message = "{ \"mtype\": 2, \"payload\": #{json_message} }"
puts "final json message: #{final_json_message}"
send ws, final_json_message
# send ws, "{ \"mtype\": 2, \"payload\": \"coucou\" }"
read ws
else
send ws, to_message(2, STDIN.gets_to_end)
read ws
end
ws.close ws.close
ws.read ws.read

92
src/websocketcspam.cr Normal file
View File

@ -0,0 +1,92 @@
require "http/web_socket"
require "option_parser"
require "./colors"
require "./utils"
require "./ws"
uri = "ws://localhost:1234/pong"
OptionParser.parse! do |parser|
parser.on "-u uri", "--uri uri", "URI" do |opturi|
uri = opturi
end
parser.on "-h", "--help", "Show this help" do
puts parser
exit 0
end
end
def read_then_print(ws : WebSocket)
m = read ws
puts "message: #{String.new(m)}"
end
def read_then_print_hexa(ws : WebSocket)
m = read ws
print_hexa(String.new(m), "#{CBLUE}Received message hexa#{CRESET}")
end
def read(ws : WebSocket)
m = ws.read
if m.nil?
raise "empty message"
end
m
end
def send_with_announce(ws : WebSocket, m : String)
puts "sending #{m}"
send ws, m
end
def send(ws : WebSocket, m : String)
ws.send m
end
def send(ws : WebSocket, m : Slice)
ws.send m
end
begin
ws = WebSocket.new(URI.parse(uri))
puts "connection done: sending pong"
ws.on_close do |socket|
puts "socket is closing"
exit 0
end
if uri.ends_with? ".JSON"
json_message = STDIN.gets_to_end
json_message = json_message.chomp
# puts "json_message: #{json_message}"
final_json_message = "{ \"mtype\": 2, \"payload\": #{json_message} }"
puts "sending 10 messages: #{final_json_message}"
10.times do |i|
send ws, final_json_message
puts "sent message #{i}"
# send ws, "{ \"mtype\": 2, \"payload\": \"coucou\" }"
end
10.times do |i|
read ws
puts "read message #{i}"
end
else
send ws, to_message(2, STDIN.gets_to_end)
read ws
end
ws.read
ws.close
rescue e
puts "Exception: #{e}"
end

View File

@ -54,7 +54,6 @@ OptionParser.parse! do |parser|
end end
end end
class InstanceStorage class InstanceStorage
property service : IPC::SwitchingService property service : IPC::SwitchingService
property switchtable : Hash(Int32, Int32) property switchtable : Hash(Int32, Int32)
@ -94,6 +93,10 @@ class InstanceStorage
@is_client = @is_client.select do |fd,v| fd != fdclient end @is_client = @is_client.select do |fd,v| fd != fdclient end
@is_json = @is_json.select do |fd,v| fd != fdclient end @is_json = @is_json.select do |fd,v| fd != fdclient end
@fd_to_websocket.select! do |fd, ws|
fd != fdclient
end
unless fdservice.nil? unless fdservice.nil?
service = @fd_to_ipcconnection[fdservice] service = @fd_to_ipcconnection[fdservice]
service.close service.close
@ -107,7 +110,8 @@ class InstanceStorage
end end
end end
server = TCPServer.new("localhost", port_to_listen) # by default, listen on any IP address
server = TCPServer.new("0.0.0.0", port_to_listen)
service = IPC::SwitchingService.new service_name service = IPC::SwitchingService.new service_name
service << server.fd service << server.fd
context = InstanceStorage.new service context = InstanceStorage.new service
@ -171,6 +175,9 @@ def websocket_client_connection(client, context : InstanceStorage)
client.send "#{headers_header}\n#{headers.to_s}\r\n" client.send "#{headers_header}\n#{headers.to_s}\r\n"
wsclient = WebSocket.new client wsclient = WebSocket.new client
wsclient.on_pong do |m|
puts "Received a pong message: #{m}"
end
context.is_client[client.fd] = true context.is_client[client.fd] = true
# listen to the client's file descriptor # listen to the client's file descriptor
@ -236,6 +243,7 @@ class IPC::Message
end end
def websocket_switching_procedure (activefd : Int, context : InstanceStorage) def websocket_switching_procedure (activefd : Int, context : InstanceStorage)
# FIXME: debugging purposes
begin begin
if context.is_client[activefd] if context.is_client[activefd]
# The client is a WebSocket on top of a TCP connection # The client is a WebSocket on top of a TCP connection
@ -244,6 +252,7 @@ def websocket_switching_procedure (activefd : Int, context : InstanceStorage)
begin begin
message = wsclient.read message = wsclient.read
# puts "RECEIVING A MESSAGE from #{activefd}: #{message}"
rescue e rescue e
puts "#{CRED}Exception (receiving a message)#{CRESET} #{e}" puts "#{CRED}Exception (receiving a message)#{CRESET} #{e}"
closing_client activefd, context closing_client activefd, context
@ -262,9 +271,29 @@ def websocket_switching_procedure (activefd : Int, context : InstanceStorage)
return return
end end
if message.is_a?(WebSocket::Ping)
# puts "#{CBLUE}This is a ping message#{CRESET}"
return
end
if message.is_a?(WebSocket::Pong)
# puts "#{CBLUE}This is a pong message#{CRESET}"
return
end
if message.is_a?(WebSocket::Close)
# puts "#{CRED}This is a close message#{CRESET}"
return
end
if message.is_a?(Slice(UInt8))
# puts "#{CRED}This is a binary message: not yet implemented#{CRESET}"
return
end
# TODO: verify this # TODO: verify this
if context.is_json[activefd] if context.is_json[activefd]
jsonmessage = JSONWSMessage.from_json String.new(message) jsonmessage = JSONWSMessage.from_json message
message = to_message jsonmessage.mtype, jsonmessage.payload message = to_message jsonmessage.mtype, jsonmessage.payload
# puts "JSON TYPE !!!" # puts "JSON TYPE !!!"
@ -281,16 +310,21 @@ def websocket_switching_procedure (activefd : Int, context : InstanceStorage)
serv = WrappedTCPFileDescriptor.new(fd: fdservice, family: Socket::Family::INET) serv = WrappedTCPFileDescriptor.new(fd: fdservice, family: Socket::Family::INET)
#serv = context.fd_to_ipcconnection[fdservice] #serv = context.fd_to_ipcconnection[fdservice]
serv.send message serv.send message
# puts "SENT MESSAGE TO SERVER: #{message}"
else else
# puts "switching: service to client" # puts "switching: service to client"
# service => client # service => client
# puts "RECEIVING A MESSAGE FROM THE SERVER #{activefd}"
fdclient = context.switchtable[activefd] fdclient = context.switchtable[activefd]
wsclient = context.fd_to_websocket[fdclient] wsclient = context.fd_to_websocket[fdclient]
serv = context.fd_to_ipcconnection[activefd] serv = context.fd_to_ipcconnection[activefd]
message = serv.read message = serv.read
# puts "RECEIVING A MESSAGE FROM THE SERVER #{activefd}: #{message}"
# puts "received message from service: #{message.to_s}" # puts "received message from service: #{message.to_s}"
if context.is_json[fdclient] if context.is_json[fdclient]
@ -316,11 +350,28 @@ def websocket_switching_procedure (activefd : Int, context : InstanceStorage)
closing_client clientfd, context closing_client clientfd, context
end end
end end
end end
# Every few seconds, the service should trigger the timer
service.base_timer = 30
service.loop do |event| service.loop do |event|
case event case event
when IPC::Event::Timer
# puts "#{CORANGE}IPC::Event::Timer#{CRESET}"
context.fd_to_websocket.each do |fd, ws|
begin
ws.ping "coucou from #{fd}"
rescue e
puts "#{CRED}Exception: #{e}#{CRESET}, already closed client #{fd}"
begin
context.remove_fd fd
rescue e
puts "#{CRED}Cannot remove #{fd} from clients: #{e}#{CRESET}"
end
end
end
when IPC::Event::Connection when IPC::Event::Connection
puts "#{CBLUE}IPC::Event::Connection: #{event.connection.fd}#{CRESET}" puts "#{CBLUE}IPC::Event::Connection: #{event.connection.fd}#{CRESET}"
when IPC::Event::Disconnection when IPC::Event::Disconnection

View File

@ -2,35 +2,71 @@
require "http" require "http"
require "http/web_socket/protocol" require "http/web_socket/protocol"
class HTTP::WebSocket
record Pong
record Ping
record Close
def read : Slice(UInt8) | String | Close | Ping | Pong | Nil
size = 0
begin
info = @ws.receive(@buffer)
rescue IO::EOFError
close
return nil
end
case info.opcode
when Protocol::Opcode::PING
@current_message.write @buffer[0, info.size]
if info.final
message = @current_message.to_s
@on_ping.try &.call(message)
pong(message) unless closed?
@current_message.clear
end
return Ping.new
when Protocol::Opcode::PONG
@current_message.write @buffer[0, info.size]
if info.final
@on_pong.try &.call(@current_message.to_s)
@current_message.clear
end
return Pong.new
when Protocol::Opcode::TEXT
message = @buffer[0, info.size]
@current_message.write message
if info.final
@on_message.try &.call(@current_message.to_s)
@current_message.clear
end
return String.new message
when Protocol::Opcode::BINARY
message = @buffer[0, info.size]
@current_message.write message
if info.final
@on_binary.try &.call(@current_message.to_slice)
@current_message.clear
end
return message
when Protocol::Opcode::CLOSE
@current_message.write @buffer[0, info.size]
if info.final
message = @current_message.to_s
@on_close.try &.call(message)
close(message) unless closed?
@current_message.clear
end
return Close.new
end
end
end
class WebSocket < HTTP::WebSocket class WebSocket < HTTP::WebSocket
getter? closed = false getter? closed = false
def read
size = 0
begin
info = @ws.receive(@buffer)
size = info.size
rescue IO::EOFError
close
return nil
end
case info.opcode
when Protocol::Opcode::TEXT
return @buffer[0..size-1]
when Protocol::Opcode::BINARY
return @buffer[0..size-1]
when Protocol::Opcode::CLOSE
begin
close
rescue e
puts "\033[31mwebsocket failed to close properly\033[00m #{e}"
end
return nil
end
end
def finalize def finalize
# puts "WrappedTCPFileDescriptor garbage collection!!" # puts "WrappedTCPFileDescriptor garbage collection!!"
# super # super