What does sidekiq need access to?

Hi there,

I'm currently trying to get the unoffical but somehow official community maintained docker images more production ready.

The main issue currently for me is that Redmine seems to expect a job queue worker that is not the defualt one used druing development or testing, something like sidekiq. Fair enough.
The images provided by https://hub.docker.com/_/redmine/ dont support that, the maintainer has raised concerns about running another process in the same container, see https://github.com/docker-library/redmine/issues/316#issuecomment-2113587316

The basic idea is to add sidekiq to the bundle as described in https://www.redmine.org/projects/redmine/wiki/SidekiqConfiguration, have the entrypoint check for config / env vars for redis and "just" start antoher instance of the same image, but with `bundle exec sidekiq` instead of `rails server -b some.binds.to.ips`as the command.

I already have this implemented and it seems to be running fine so far, there are just some things I'd like to improve / understand a bit better before maybe submitting a PR:

Currently, the second instance of the image, the on running `bundle exec sidekiq`, is basically checking the same configs as the main instance, its also checking to see if the database / sql connection is correctly configured.
My question is: Does sidekiq even need access to the database? are the jobs run on it read and write from it, or is everything that a job needs and does contained within the ruby code and redis?

If sidekiq doesnt need access to the DB, that would allow me to clean up some things in the entrypoint further.

Thanks for any hints and help in advance

Replies (5)

RE: What does sidekiq need access to? - Added by Holger Just 6 months ago

The job workers (e.g sidekiq workers) definitely need access to the Redmine database. The job definitions contain only enough information to get the jobs started but almost all jobs also require additional database information to run. In fact, even starting the sidekiq workers will likely fail without access to the database.

With that being said, if you want to reduce the number of dependencies, you may want to investigate delayed_job and delayed_job_active_record instead if sidekiq. Here, the jobs are written to a database table instead of a Redis server. This works fine with Redmine too and completely removes the dependency in Redis.

RE: What does sidekiq need access to? - Added by Bernd Konrad 6 months ago

Reducing the number of dependencies is not really an issue here.

Thx for the information on the need for database connections though, thats all the info I need :)

RE: What does sidekiq need access to? - Added by Bernd Konrad 5 months ago

So, I managed to get everything working, i provided some axmaples here: https://github.com/DerDummePunkt/redmine_docker_compose_sidekiq

What cost me a bit of time though was the config for sidekiq.

By default, sidekiq just listens to a queue named "default". Redmine puts tasks in a queue named "mailers" (and maybe others).
There is alsoa set key named "queues" in redis that contains the entry "mailers", I would assume thats where redmine expects sidekiq to look.
By default, sidekiq neither looks a the "queues" set key, nor does it listen to "mailers".
I had to configure sidekiq to explicitly listen to the queue "mailers", else redmine would happily push tasks on the queue, but sidekiq would just sit there and do nothing.

This fact was in the wiki article on the sidekiq config until January 2024, but someone removed it:
https://www.redmine.org/projects/redmine/wiki/SidekiqConfiguration/diff?utf8=%E2%9C%93&commit=Unterschiede+anzeigen&version=5&version_from=4

How does the wiki here work in terms of permissions? I havent found any information on that.
Can any registered member make changes? Do they get reviewed? Can I add that paragraph back in?

RE: What does sidekiq need access to? - Added by Harry Lime 8 days ago

Hi Bernd - did you get any further on this? My guess is that people 'normally' start sidekiq with -q default -q mailers (see https://www.redmine.org/boards/2/topics/69656, for example), in which case the additional config is unnecessary. This should, of course, be in the wiki.

RE: What does sidekiq need access to? - Added by Bernd Konrad 7 days ago

Hey Harry,

What I'm currently running is what you find at https://github.com/DerDummePunkt/redmine_docker_compose_sidekiq, it works for me without any issues, mails work as intended

Basically, would I did is:

modified the docker image provided via https://github.com/docker-library/redmine and installed sidekiq in the image
run two services based on the image, on running the rails server, the other running sidekiq
sidekiq is just listening to the "mailers" queue, not to "default" * as far as i can tell, redmine never puts anything in to a queue named "default"

I had planned on working with the maintainer for the docker community images to upstream my changes, unfortunately they took some months to reply, by now I don't have enough time to deal that upsteam process anymore, I might still tackle it, if i find a quiet weekend, but if the community maintainer takes another 4 months to reply to anything, I'm probably abandoning those plans ;)

(1-5/5)

Project

General

Profile

Redmine