122 lines
2.8 KiB
Markdown
122 lines
2.8 KiB
Markdown
# mechanize.cr
|
|
|
|
[](https://github.com/Kanezoh/mechanize.cr/actions/workflows/crystal.yml)
|
|
|
|
This project is inspired by Ruby's [mechanize](https://github.com/sparklemotion/mechanize).
|
|
The purpose is to cover all the features of original one.
|
|
|
|
[API Documentation](https://kanezoh.github.io/mechanize.cr/)
|
|
|
|
## Installation
|
|
|
|
1. Add the dependency to your `shard.yml`:
|
|
|
|
```yaml
|
|
dependencies:
|
|
mechanize:
|
|
github: Kanezoh/mechanize.cr
|
|
```
|
|
|
|
2. Run `shards install`
|
|
|
|
## Usage
|
|
|
|
### GET request
|
|
|
|
```crystal
|
|
require "mechanize"
|
|
|
|
agent = Mechanize.new
|
|
page = agent.get("http://example.com/")
|
|
|
|
puts page.code # => 200
|
|
puts page.body # => html
|
|
puts page.title # => Example Domain
|
|
```
|
|
|
|
|
|
### POST request
|
|
|
|
You can also send post request with data.
|
|
|
|
```crystal
|
|
require "mechanize"
|
|
|
|
agent = Mechanize.new
|
|
query = {"foo" => "foo_value", "bar" => "bar_value"}
|
|
page = agent.post("http://example.com/", query: query)
|
|
# => request body is foo=foo_value&bar=bar_value
|
|
```
|
|
|
|
### add query params, request_headers
|
|
|
|
You can add any query parameters and headers to requests.
|
|
|
|
```crystal
|
|
require "mechanize"
|
|
|
|
agent = Mechanize.new
|
|
agent.request_headers = HTTP::Headers{"Foo" => "Bar"}
|
|
params = {"hoge" => "hoge"}
|
|
page = agent.get("http://example.com/", params: params)
|
|
# The actual URL is http://example.com/?hoge=hoge
|
|
```
|
|
|
|
### fill and submit form
|
|
|
|
You can fill and submit form by using `field_with` and `submit`. It enables you to scrape web pages requiring login.
|
|
|
|
```crystal
|
|
require "mechanize"
|
|
|
|
agent = Mechanize.new
|
|
page = agent.get("#{web page contains login form}")
|
|
form = page.forms[0]
|
|
form.field_with("email").value = "tester@example.com"
|
|
form.field_with("password").value = "xxxxxx"
|
|
agent.submit(form)
|
|
|
|
agent.get("#{web page only logged-in users can see}"
|
|
```
|
|
|
|
### search node
|
|
|
|
You can use css selector to search html nodes by using `#css` method.
|
|
This method is from [lexbor](https://github.com/kostya/lexbor), so if you want to explore more, please refer the repository.
|
|
|
|
```crystal
|
|
puts page.css("h1").first.inner_text
|
|
```
|
|
|
|
### Logging
|
|
|
|
For activation, simply setup the log to `:debug` level
|
|
|
|
```crystal
|
|
Log.setup("mechanize", :debug)
|
|
```
|
|
|
|
### Basic auth
|
|
|
|
You can access a page which is protected by basic auth, setting username and password for the url.
|
|
|
|
```crystal
|
|
agent = Mechanize.new
|
|
agent.add_auth("http://example.com", "username", "password")
|
|
agent.get("http://example.com")
|
|
```
|
|
|
|
|
|
## Contributing
|
|
|
|
1. Fork it (<https://github.com/Kanezoh/mechanize.cr/fork>)
|
|
2. Create your feature branch (`git checkout -b my-new-feature`)
|
|
3. Commit your changes (`git commit -am 'Add some feature'`)
|
|
4. Push to the branch (`git push origin my-new-feature`)
|
|
5. Create a new Pull Request
|
|
|
|
## Contributors
|
|
|
|
- [Kanezoh](https://github.com/Kanezoh) - creator and maintainer
|
|
- [mamantoha](https://github.com/mamantoha) - contributor
|