Add Crystal Digest overrides

Uses mmap with guard pages

New SecurityLevel: :strong

Adjust security levels <=> secret class mapping

README.md

@@ -8,12 +8,24 @@ Secrets hold sensitive information
 
 The Secret interface manages limited time access to a secret and securely erases the secret when no longer needed.
 
-Multiple `Secret` classes exist.  Most of the time you shouldn't need to change the `Secret` type - the cryptographic library should have sane defaults.
+Multiple `Secret` classes exist.  Most of the time you shouldn't need to change the `Secret` type. The cryptographic library should have sane defaults.
 If you have a high security or high performance application see [which secret type should I use?](https://didactic-drunk.github.io/crypto-secret.cr/main/Crypto/Secret.html)
 
+
+## What attacks does a Secret protect against?
+
+* Timing attacks when comparing secrets by overriding `==`
+* Leaking data in to logs by overriding `inspect`
+* Wiping memory when the secret is no longer in use
+
+### Provided secret classes
+* `Crypto::Secret::Guarded` - Guard pages, mprotect, doesn't appear in core dumps (os dependent)
+* `Crypto::Secret::Bidet` - Wipe only.  Low overhead.
+* `Crypto::Secret::Not` - It's not secret.  Doesn't wipe and no additional protection.
+* `Crypto::Secret::Todo` - Uses mlock, mprotect and canaries in future versions
+
 Secret providers may implement additional protections via:
-* `#noaccess`, `#readonly` or `#readwrite`
+* `#noaccess`, `#readonly` or `#readwrite` via `mprotect`
-* Using [mprotect]() to control access
 * Encrypting the data when not in use
 * Deriving keys on demand from a HSM
 * Preventing the Secret from entering swap ([mlock]())
@@ -28,7 +40,7 @@ Secret providers may implement additional protections via:
    ```yaml
    dependencies:
      crypto-secret:
-       github: didactic-drunk/crypto-secret
+       github: didactic-drunk/crypto-secret.cr
    ```
 
 2. Run `shards install`
@@ -44,12 +56,11 @@ Secret providers may implement additional protections via:
 ```crystal
 require "crypto-secret/bidet"
 
-# Bidet is a minimal but fast secret implementation
+secret = Crypto::Secret.for(32, :secret_key)
-secret = Crypto::Secret::Bidet.new 32
 # Don't forget to wipe!
 secret.wipe do
   secret.readonly do |slice|
-    # May only read slice
+    # May only read from slice
   end
   secret.readwrite do |slice|
     # May read or write to slice
@@ -78,10 +89,28 @@ secret = Crypto::Secret::Bidet.move_from slice # erases slice
 # or
 secret = Crypto::Secret::Bidet.copy_from slice
 # or
-secret = Crypto::Secret::Bidet size_in_bytes
+secret = Crypto::Secret::Bidet.new size_in_bytes
 secret.move_from slice
 ```
 
+### Optionally change the security level
+
+The default should be sufficient for most applications.  Do not change unless you have special needs.
+
+Password managers or cryptocurrency wallets may prefer :strong or :paranoid.
+
+Blockchain verifiers or apps that only handle high volume public info may prefer :lax.
+
+```crystal
+# Choose one
+Crypto::Secret::Config.setup :paranoid
+Crypto::Secret::Config.setup :strong
+#Crypto::Secret::Config.setup :default # automatic
+Crypto::Secret::Config.setup :lax
+```
+
+See [#setup](https://didactic-drunk.github.io/crypto-secret.cr/main/Crypto/Secret/Config.html) for further information.
+
 ## What is a Secret?
 
 <strike>Secrets are Keys</strike>
@@ -93,8 +122,8 @@ That's complicated and specific to the application.  Some examples:
 
 Not secrets:
 
-* Digest output.  Except when used for key derivation, then it's a Secret, including the Digest state
+* `Digest` output.  Except when used for key derivation, then it's a Secret, including the Digest state
-* IO::Memory or writing a file.  Except when the file is a password vault, cryptocurrency wallet, encrypted mail/messages, goat porn, maybe "normal" porn, sometimes scat porn, occassionally furry, not people porn
+* `IO::Memory` or writing a file.  Except when the file is a password vault, cryptocurrency wallet, encrypted mail/messages, goat porn, maybe "normal" porn, sometimes scat porn, occassionally furry, not people porn
 
 ## Why?
 
@@ -128,9 +157,9 @@ end
 key = ...another Secret...
 encrypted_str = File.read("filename")
 decrypted_size = encrypted_str.bytesize - mac_size
-file_secret = Crypto::Secret::Large.new decrypted_size
+file_secret = Crypto::Secret.for(decrypted_size, :data)
 file_secret.wipe do
-  file_secrets.readwrite do |slice|
+  file_secret.readwrite do |slice|
     decrypt(key: key, src: encrypted_str, dst: slice)
 
     # Do something with file contents in slice
@@ -159,26 +188,13 @@ Example:
 class SimplifiedEncryption
   # Allow users of your library to provide their own Secret key.  Also provide a sane default.
   def encrypt(data : Bytes | String, key : Secret? = nil) : {Secret, Bytes}
-    key ||= Crypto::Secret::Default.random
+    key ||= Crypto::Secret.for(key_size, :secret_key)
     ...
     {key, encrypted_slice}
   end
 end
 ```
 
-## What attacks does a Secret protect against?
-
-* Timing attacks when comparing secrets by overriding `==`
-* Leaking data in to logs by overriding `inspect`
-* Wiping memory when the secret is no longer in use
-
-Each implementation may add additional protections
-
-* `Crypto::Secret::Key` - May use mlock, mprotect and canaries in future versions
-* `Crypto::Secret::Large` - May use mprotect in future versions
-* `Crypto::Secret::Not` - It's not secret.  Doesn't wipe and no additional protection.
-
-
 ## Other languages/libraries
 
 * rust: [secrets](https://github.com/stouset/secrets/)
@@ -191,7 +207,7 @@ Each implementation may add additional protections
 
 **Only intended for use by crypto library authors**
 
-```
+```crystal
 class MySecret < Crypto::Secret
   # Choose one
   include Crypto::Secret::Stateless
@@ -213,7 +229,8 @@ class MySecret < Crypto::Secret
     # return the size
   end
 
-  # optionally override [noaccess, readonly, readwrite]
+  # if Stateful provide [noaccess_impl, readonly_impl, readwrite_impl]
+
   # optionally override (almost) any other method with an implementation specific version
 end
 

shard.yml

@@ -6,4 +6,9 @@ authors:
 
 crystal: ">= 0.37"
 
+dependencies:
+  mmap:
+    github: crystal-posix/mmap.cr
+    version: ">= 0.4.0"
+
 license: MIT

spec/crypto_secret_spec.cr

@@ -4,12 +4,29 @@ require "../src/crypto-secret"
 
 test_secret_class Crypto::Secret::Not
 test_secret_class Crypto::Secret::Bidet
+test_secret_class Crypto::Secret::Guarded
 
 describe Crypto::Secret do
   it ".for" do
-    [:kgk, :key, :data, :not].each do |sym|
+    [:kgk, :secret_key, :public_key, :data, :not].each do |sym|
-      secret = Crypto::Secret.for sym, 2
+      secret = Crypto::Secret.for 2, sym
       secret.bytesize.should eq 2
     end
   end
+
+  it ".for fallback" do
+    secret = Crypto::Secret.for 2, :a, :b, :not
+    secret.bytesize.should eq 2
+  end
+
+  it ".for missing" do
+    expect_raises(KeyError) do
+      Crypto::Secret.for 2, :a
+    end
+  end
+
+  it ".random" do
+    secret = Crypto::Secret.random 2, :a, :b, :not
+    secret.bytesize.should eq 2
+  end
 end

src/crypto-secret/config.cr

@@ -1,10 +1,11 @@
 require "./not"
 require "./bidet"
+require "./guarded"
 
 {% if @type.has_constant?("Sodium") %}
   CRYPTO_SECRET_KEY_CLASS = Sodium::SecureBuffer
 {% else %}
-  CRYPTO_SECRET_KEY_CLASS = Crypto::Secret::Bidet
+  CRYPTO_SECRET_KEY_CLASS = Crypto::Secret::Guarded
 {% end %}
 
 module Crypto::Secret::Config
@@ -12,26 +13,34 @@ module Crypto::Secret::Config
   USES = Hash(Symbol, Secret.class).new
 
   enum SecurityLevel
+    # mlocks everything (including data)
     Paranoid
+    # wipes everything
+    Strong
+    # balance between performance and wiping
     Default
+    # performance
     Lax
     #    None
   end
 
-  def self.setup(level : SecurityLevel = SecurityLevel::Default) : Nil
+  def self.setup(level : SecurityLevel = :default) : Nil
-    register_use Not, :not
+    register_use Not, :not, :public_key
 
     case level
     in SecurityLevel::Paranoid
       register_use Bidet, :not
-      register_use CRYPTO_SECRET_KEY_CLASS, :kgk, :key, :data
+      register_use Guarded, :public_key
+      register_use CRYPTO_SECRET_KEY_CLASS, :kgk, :secret_key, :data
+    in SecurityLevel::Strong
+      register_use Bidet, :not, :public_key
+      register_use Crypto::Secret::Guarded, :data
+      register_use CRYPTO_SECRET_KEY_CLASS, :kgk, :secret_key
     in SecurityLevel::Default
       register_use Crypto::Secret::Bidet, :data
-      register_use CRYPTO_SECRET_KEY_CLASS, :kgk, :key
+      register_use CRYPTO_SECRET_KEY_CLASS, :kgk, :secret_key
     in SecurityLevel::Lax
-      register_use Bidet, :kgk, :key, :data
+      register_use Bidet, :kgk, :secret_key, :data
-      #      in SecurityLevel::None
-      #        register_use Not, :kgk, :key, :data
     end
   end
 

src/crypto-secret/crystal.cr

@@ -0,0 +1,13 @@
+abstract class Digest
+  def update(data : Crypto::Secret)
+    data.readonly do |slice|
+      update slice
+    end
+  end
+
+  def final(data : Crypto::Secret)
+    data.readwrite do |slice|
+      final slice
+    end
+  end
+end

src/crypto-secret/guarded.cr

@@ -0,0 +1,47 @@
+require "./stateful"
+require "mmap"
+
+abstract class Crypto::Secret
+  # * Wipes on finalize but should not be relied on
+  # * Not locked in memory
+  # * Access protected
+  # * Guard pages
+  # * Won't appear in core dumps (some platforms)
+  class Guarded < Secret
+    include Stateful
+
+    protected getter buffer_bytesize : Int32
+    @dregion : Mmap::SubRegion
+    @data : Mmap::SubRegion
+
+    def initialize(size : Int32)
+      ps = Mmap::PAGE_SIZE
+      pages = (size.to_f / ps).ceil + 2
+      msize = pages * ps
+
+      @buffer_bytesize = size
+
+      @mmap = Mmap::Region.new(msize)
+      @mmap[0, ps].guard_page
+      @mmap[(pages - 1) * ps, ps].guard_page
+
+      @dregion = @mmap[ps, (pages - 2) * ps]
+      @dregion.crypto_key
+      @data = @dregion[0, size]
+    end
+
+    protected def readwrite_impl : Nil
+      @dregion.readwrite
+    end
+
+    protected def readonly_impl : Nil
+      @dregion.readonly
+    end
+
+    protected def noaccess_impl : Nil
+      @dregion.noaccess
+    end
+
+    delegate_to_slice @data
+  end
+end

src/crypto-secret/secret.cr

@@ -1,11 +1,13 @@
 require "./lib"
 require "./class_methods"
+require "./crystal"
 
 # Interface to hold sensitive information (often cryptographic keys)
 #
 # ## Which class should I use?
-# * `Crypto::Secret::Key` - Use with small (<= 4096 bytes) keys
+# * `Crypto::Secret::Todo` - Use with small (<= 4096 bytes) keys
-# * `Crypto::Secret::Large` - Use for decrypted data that may stress mlock limits
+# * `Crypto::Secret::Guarded` - Use for decrypted data that may stress mlock limits
+# * `Crypto::Secret::Bidet` - Wipe only with no other protection.  General use and fast.
 # * `Crypto::Secret::Not` - Only use when you're sure the data isn't secret.  0 overhead.  No wiping.
 #
 # Other shards may provide additional `Secret` types ([sodium.cr](https://github.com/didactic-drunk/sodium.cr))
@@ -31,22 +33,36 @@ abstract class Crypto::Secret
     Readwrite
   end
 
-  extend ClassMethods
+  macro inherited
+    extend ClassMethods
+  end
 
   def self.new(size : Int32)
     raise NotImplementedError.new("workaround for lack of `abstract def self.new`")
   end
 
-  def self.for(secret : Crypto::Secret) : Crypto::Secret
+  #  def self.random(size : Int32, *uses : Symbol, *, random = Random::Secure) : Crypto::Secret
+  def self.random(size : Int32, *uses : Symbol, **options) : Crypto::Secret
+    rand = options[:random]? || Random::Secure
+    for(size, *uses).random(random: rand)
+  end
+
+  def self.for(size : Int32, *uses : Symbol) : Crypto::Secret
+    for(*uses).new(size)
+  end
+
+  def self.for(size : Int32, secret : Crypto::Secret) : Crypto::Secret
+    raise ArgumentError.new("") unless size == secret.bytesize
     secret
   end
 
-  def self.for(use : Symbol, size : Int32) : Crypto::Secret
+  def self.for(*uses : Symbol) : Crypto::Secret.class
-    for(use).new(size)
+    uses.each do |use|
-  end
+      if klass = Config::USES[use]?
-
+        return klass
-  def self.for(use : Symbol) : Crypto::Secret.class
+      end
-    Config::USES[use]
+    end
+    raise KeyError.new("missing #{uses}, have #{Config::USES.keys}")
   end
 
   # For debugging.  Leaks the secret
@@ -85,9 +101,9 @@ abstract class Crypto::Secret
   end
 
   # Fills `Secret` with secure random data
-  def random : self
+  def random(random = Random::Secure) : self
     readwrite do |slice|
-      Random::Secure.random_bytes slice
+      random.random_bytes slice
     end
     self
   end

src/crypto-secret/stateful.cr

@@ -1,122 +1,122 @@
 require "./secret"
 
-  # Development guide:
+# Development guide:
-  # 1. Create your initialize method and optionally allocate memory
+# 1. Create your initialize method and optionally allocate memory
-  # 2. Create a finalize method to deallocate memory if necessary
+# 2. Create a finalize method to deallocate memory if necessary
-  # 3. Fill in the missing abstract methods
+# 3. Fill in the missing abstract methods
-  # 4. Optionally override any included methods (especially wipe_impl if the secret is not held in the provided slice)
+# 4. Optionally override any included methods (especially wipe_impl if the secret is not held in the provided slice)
-  # 5. Provide and test a dup method or raise on dup if not possible
+# 5. Provide and test a dup method or raise on dup if not possible
-  #
+#
-  # When state changes are required (such as using #noaccess) and the buffer is accessed from multiple threads wrap each #readonly/#readwrite block in a lock.
+# When state changes are required (such as using #noaccess) and the buffer is accessed from multiple threads wrap each #readonly/#readwrite block in a lock.
 module Crypto::Secret::Stateful
-    @state = State::Readwrite
+  @state = State::Readwrite
-    @pre_wipe_state = State::Readwrite
+  @pre_wipe_state = State::Readwrite
 
-    # Temporarily make buffer readwrite within the block returning to the prior state on exit.
+  # Temporarily make buffer readwrite within the block returning to the prior state on exit.
-    # WARNING: Not thread safe unless this object is **readwrite**
+  # WARNING: Not thread safe unless this object is **readwrite**
-    def readwrite
+  def readwrite
-      with_state State::Readwrite do
+    with_state State::Readwrite do
-        to_slice do |slice|
+      to_slice do |slice|
-          yield slice
+        yield slice
-        end
       end
     end
-
-    # Marks a region allocated as readable and writable
-    # WARNING: Not thread safe
-    def readwrite : Secret
-      raise Error::KeyWiped.new if @state == State::Wiped
-      readwrite_impl
-      @state = State::Readwrite
-      self
-    end
-
-    # Temporarily make buffer readonly within the block returning to the prior state on exit.
-    # WARNING: Not thread safe unless this object is readonly or readwrite
-    def readonly(& : Bytes -> U) forall U
-      with_state State::Readonly do
-        to_slice do |slice|
-          yield slice
-        end
-      end
-    end
-
-    # Marks a region allocated using sodium_malloc() or sodium_allocarray() as read-only.
-    # WARNING: Not thread safe
-    def readonly : Secret
-      raise Error::KeyWiped.new if @state == State::Wiped
-      readonly_impl
-      @state = State::Readonly
-      self
-    end
-
-    # Makes a region inaccessible. It cannot be read or written, but the data are preserved.
-    # WARNING: Not thread safe
-    def noaccess : Secret
-      raise Error::KeyWiped.new if @state == State::Wiped
-      noaccess_impl
-      @state = State::Noaccess
-      self
-    end
-
-    def reset
-      case @state
-      when State::Wiped; set_state @pre_wipe_state
-      else
-        wipe_impl
-      end
-    end
-
-    # WARNING: Not thread safe
-    # Kept public for .dup
-    # :nodoc:
-    def set_state(new_state : State)
-      return if @state == new_state
-
-      case new_state
-      when State::Readwrite; readwrite
-      when State::Readonly ; readonly
-      when State::Noaccess ; noaccess
-      when State::Wiped    ; raise Error::KeyWiped.new
-      else
-        raise Error::InvalidStateTransition.new("can't change to #{new_state}")
-      end
-    end
-
-    # WARNING: Only thread safe when current state >= requested state
-    private def with_state(new_state : State)
-      old_state = @state
-      # Only change when new_state needs more access than @state.
-      if old_state >= new_state
-        yield
-      else
-        begin
-          set_state new_state
-          yield
-        ensure
-          set_state old_state
-        end
-      end
-    end
-
-    # WARNING: Not thread safe
-    def wipe
-      return if @state == State::Wiped
-      @pre_wipe_state = @state
-      readwrite do |slice|
-        wipe_impl slice
-      end
-      noaccess_impl
-      @state = State::Wiped
-    end
-
-    def dup
-      super.tap do |obj|
-        obj.set_state @state
-      end
-    end
-
-    protected abstract def readwrite_impl : Nil
-    protected abstract def readonly_impl : Nil
-    protected abstract def noaccess_impl : Nil
   end
+
+  # Marks a region allocated as readable and writable
+  # WARNING: Not thread safe
+  def readwrite : Secret
+    raise Error::KeyWiped.new if @state == State::Wiped
+    readwrite_impl
+    @state = State::Readwrite
+    self
+  end
+
+  # Temporarily make buffer readonly within the block returning to the prior state on exit.
+  # WARNING: Not thread safe unless this object is readonly or readwrite
+  def readonly(& : Bytes -> U) forall U
+    with_state State::Readonly do
+      to_slice do |slice|
+        yield slice
+      end
+    end
+  end
+
+  # Marks a region allocated using sodium_malloc() or sodium_allocarray() as read-only.
+  # WARNING: Not thread safe
+  def readonly : Secret
+    raise Error::KeyWiped.new if @state == State::Wiped
+    readonly_impl
+    @state = State::Readonly
+    self
+  end
+
+  # Makes a region inaccessible. It cannot be read or written, but the data are preserved.
+  # WARNING: Not thread safe
+  def noaccess : Secret
+    raise Error::KeyWiped.new if @state == State::Wiped
+    noaccess_impl
+    @state = State::Noaccess
+    self
+  end
+
+  def reset
+    case @state
+    when State::Wiped; set_state @pre_wipe_state
+    else
+      wipe_impl
+    end
+  end
+
+  # WARNING: Not thread safe
+  # Kept public for .dup
+  # :nodoc:
+  def set_state(new_state : State)
+    return if @state == new_state
+
+    case new_state
+    when State::Readwrite; readwrite
+    when State::Readonly ; readonly
+    when State::Noaccess ; noaccess
+    when State::Wiped    ; raise Error::KeyWiped.new
+    else
+      raise Error::InvalidStateTransition.new("can't change to #{new_state}")
+    end
+  end
+
+  # WARNING: Only thread safe when current state >= requested state
+  private def with_state(new_state : State)
+    old_state = @state
+    # Only change when new_state needs more access than @state.
+    if old_state >= new_state
+      yield
+    else
+      begin
+        set_state new_state
+        yield
+      ensure
+        set_state old_state
+      end
+    end
+  end
+
+  # WARNING: Not thread safe
+  def wipe
+    return if @state == State::Wiped
+    @pre_wipe_state = @state
+    readwrite do |slice|
+      wipe_impl slice
+    end
+    noaccess_impl
+    @state = State::Wiped
+  end
+
+  def dup
+    super.tap do |obj|
+      obj.set_state @state
+    end
+  end
+
+  protected abstract def readwrite_impl : Nil
+  protected abstract def readonly_impl : Nil
+  protected abstract def noaccess_impl : Nil
+end