Httrack Web Spider Login Problem; Cookie Requirements of Redmine / Rails

Added by Kurt Miebach about 13 years ago

I am trying to spider some projects on a redmine installation with webhttrack for offline browsing.

The first request with httrack succeeds, but on the second request the spider is redirected to the redmine login page at /login, so I suspect the session is destroyed somehow.

For the first request to succeed I provide a cookies.txt file to httrack which looks like this:

# HTTrack Website Copier Cookie File
# This file format is compatible with Netscape cookies
mydomain.com    FALSE    /    FALSE    0    _redmine_session    BAh7CzoMdXNlcl9pZGkJOhBfY3NyZl90b2tlbiIxZ3FtSHJjUzdrUFBUa2dhamRQUGo2VHhlQ2tTblRxL2c5NUQ2QWZ2alo5VT06CnF1ZXJ5ewk6EWNvbHVtbl9uYW1lczA6DWdyb3VwX2J5MDoPcHJvamVjdF9pZGkUOgxmaWx0ZXJzewYiDnN0YXR1c19pZHsHOg1vcGVyYXRvciIGbzoLdmFsdWVzWwYiACIWaXNzdWVzX2luZGV4X3NvcnQiFHVwZGF0ZWRfb246ZGVzXzoPc2Vzc2lvbl9pZCIlNmFjNzdhOTFhZDMyYTQ5NjIwNTBiN2YwMzI4ZjVhNjUiCmZsYXNoSUM6J0FjdGlvbkNvbnRyb2xsZXI6OkZsYXNoOjpGbGFzaEhhc2h7AAY6CkB1c2VkewA%3D--65ab48cc09effcd03ffe45a134735bbaedc48101
mydomain.com    FALSE    /    FALSE    1999999999    autologin    fcff1f81350bec116bb12185626cff3a18d8ec0f

I have exported the cookies from a firefox session where I am already logged in.

I have both tried "accept cookies" in httrack set to true or false.

I wonder why I am logged out after the first request?

Does redmine change cookies or resend changed cookies after each request? What else could be a solution?

Replies (3)

RE: Httrack Web Spider Login Problem; Cookie Requirements of Redmine / Rails - Added by Etienne Massip about 13 years ago

Yes, by default, rails (and so redmine) stores session data in "_redmine_session" cookie, so the cookie value change at every request.

RE: Httrack Web Spider Login Problem; Cookie Requirements of Redmine / Rails - Added by Etienne Massip about 13 years ago

You could try to use another store, you'll find some examples with Google : http://www.quarkruby.com/2007/10/21/sessions-and-cookies-in-ruby-on-rails#sstorage

RE: Httrack Web Spider Login Problem; Cookie Requirements of Redmine / Rails - Added by Kurt Miebach about 13 years ago

I wrote a long reply, but I got "iinvalid form auth token" and now it is gone.

In short:

I am using redmine 1.0.2

I tried this in envirponment.rb:

config.action_controller.cache_store = :file_store

but the cookies still changed on each request.

This setting resulted in a NameError:

config.action_controller.cache_store = :PStore

Also this one:

config.action_controller.cache_store = :active_record_store

What is the correct setting?

(1-3/3)

Project

General

Profile

Redmine