Patch #16069

Allow configuration of API limit instead of hardcoding at 100

Added by Domingo Galdos about 7 years ago. Updated 20 days ago.

Status:NewStart date:
Priority:NormalDue date:
Assignee:-% Done:

0%

Category:REST API
Target version:5.0.0

Description

The below tested patch makes the previously harcoded limit of 100 items in a RESTful API response configurable in the Administration GUI. The default value is still 100. This is based on 2.4.1.stable.

This is quite useful since it is nice and simple to just get the full response in a single page rather than having to traverse pagination, and a limit of a bit more than 100 is still very performant for many sites -- the limit beyond which performance starts to degrade really depends on the particular site's usage, so should be controllable by the local administrator.

Index: app/controllers/application_controller.rb
===================================================================
--- app/controllers/application_controller.rb       (.../trunk/app) (revision 13385)
+++ app/controllers/application_controller.rb       (.../branches/code-8112/app)    (revision 13389)
@@ -519,8 +519,8 @@
     limit = options[:limit].to_i
     if limit < 1
       limit = 25
-    elsif limit > 100
-      limit = 100
+    elsif limit > Setting.api_limit.to_i
+      limit = Setting.api_limit.to_i
     end
     if offset.nil? && options[:page].present?
       offset = (options[:page].to_i - 1) * limit
Index: app/views/settings/_general.html.erb
===================================================================
--- app/views/settings/_general.html.erb    (.../trunk/app) (revision 13385)
+++ app/views/settings/_general.html.erb    (.../branches/code-8112/app)    (revision 13389)
@@ -24,6 +24,8 @@

 <p><%= setting_select :wiki_compression, [['Gzip', 'gzip']], :blank => :label_none %></p>

+<p><%= setting_text_field :api_limit, :size => 6 %></p>
+
 <p><%= setting_text_field :feeds_limit, :size => 6 %></p>

 <p><%= setting_text_field :file_max_size_displayed, :size => 6 %> <%= l(:"number.human.storage_units.units.kb") %></p>
Index: config/settings.yml
===================================================================
--- config/settings.yml        (revision 13387)
+++ config/settings.yml        (revision 13389)
@@ -73,6 +73,9 @@
   default: localhost:3000
 protocol:
   default: http
+api_limit:
+  format: int
+  default: 100
 feeds_limit:
   format: int
   default: 15
Index: config/locales/en.yml
===================================================================
--- config/locales/en.yml      (revision 13387)
+++ config/locales/en.yml      (revision 13389)
@@ -352,6 +352,7 @@
   setting_host_name: Host name and path
   setting_text_formatting: Text formatting
   setting_wiki_compression: Wiki history compression
+  setting_api_limit: Maximum number of items returned in a JSON/XML API response
   setting_feeds_limit: Maximum number of items in Atom feeds
   setting_default_projects_public: New projects are public by default
   setting_autofetch_changesets: Fetch commits automatically

0001-domingo-galdos.patch Magnifier (1.93 KB) vzvu 3k6k, 2020-10-24 09:56

0002-add-test.patch Magnifier (658 Bytes) vzvu 3k6k, 2020-10-24 09:56

0004-simplify-label.patch Magnifier (643 Bytes) vzvu 3k6k, 2020-10-24 09:56

0003-move-to-api-tab.patch Magnifier (905 Bytes) vzvu 3k6k, 2020-10-24 09:56

0005-quickfix-allow-less-than-25.patch Magnifier (544 Bytes) vzvu 3k6k, 2020-10-24 09:56

Add-validation-to-api-limit.patch Magnifier (1.41 KB) Mizuki ISHIKAWA, 2020-10-26 08:08


Related issues

Duplicated by Redmine - Feature #7539: Rest-API limit of items not 100, but changeable by the ad... Closed 2011-02-04
Duplicated by Redmine - Defect #25555: Make api paging limit configurable Closed
Duplicated by Redmine - Feature #26553: Overly restricted number of ReST results Closed
Duplicated by Redmine - Feature #33526: Add possibility to configure limit used to retrieve objec... Closed

History

#1 Updated by Domingo Galdos over 6 years ago

Is there anything I can do to help move this patch forward?

#2 Updated by Anton Statutov almost 6 years ago

+1

#3 Updated by V Lindell almost 6 years ago

+1

#4 Updated by Toshi MARUYAMA over 5 years ago

  • Target version set to 3.2.0

#5 Updated by Go MAEDA over 5 years ago

The user should not be able to set the value of API limit smaller than 100.
Probably some clients suppose that the value is always 100, smaller limit may affect compatibility with such clients.

#6 Updated by Jens Peter Kempkes over 5 years ago

We would also appreciate higher values than 100. Some of our tools need to load all items visible to a user and therefore need to split the requests.

#7 Updated by Sebastian Paluch over 5 years ago

+1

#8 Updated by Jean-Philippe Lang over 5 years ago

  • Target version changed from 3.2.0 to Candidate for next major release

#9 Updated by Jens Peter Kempkes over 5 years ago

Too sad, this patch was shifted.
What is the reason?

We're really waiting for this patch and all seems to be done. Am I wrong?

#10 Updated by Niklaus Jordi over 5 years ago

+1

#11 Updated by Go MAEDA about 5 years ago

  • Duplicated by Feature #7539: Rest-API limit of items not 100, but changeable by the administrator added

#12 Updated by Dam Dam almost 5 years ago

+1

Hi,
This request was posted 2 years ago, so what's up?

#13 Updated by Stephane Evr over 4 years ago

+1

#14 Updated by Go MAEDA about 4 years ago

  • Duplicated by Defect #25555: Make api paging limit configurable added

#15 Updated by George Brooke about 4 years ago

Is there anything wrong/non-ideal with this patch which means that it has not been integrated?

#16 Updated by Bohdan Artemchuk almost 4 years ago

+1

#17 Updated by Toshi MARUYAMA over 3 years ago

  • Description updated (diff)

#18 Updated by Toshi MARUYAMA over 3 years ago

  • Duplicated by Feature #26553: Overly restricted number of ReST results added

#19 Updated by Michael Gerz over 3 years ago

What happened to this patch? Nothing?

#21 Updated by Гордеев Алексей over 2 years ago

Silence... :(

#22 Updated by Yuuki NARA about 2 years ago

+1

I think that it should be changeable by the responsibility of the system administrator.

There are many configuration items in the world system that may cause troubles due to setting change contents.
Since there are cases where it is necessary to change these items, it is made public after clearly stating that there is a risk when changing.

It is enough to specify the following points.

If you change the default value, there is a possibility of trouble in the operation of REST using application.
The operation verification responsibility is on Redmine system administrator who changed.

#23 Updated by C F about 2 years ago

+1

#24 Updated by Gianni Cavallotto about 2 years ago

+1

#25 Updated by int redmine over 1 year ago

+1

#26 Updated by Meiki Neumann over 1 year ago

+1

#27 Updated by r okui over 1 year ago

+1

#28 Updated by Cédric Berger over 1 year ago

+1

#29 Updated by Mischa The Evil 11 months ago

  • Duplicated by Feature #33526: Add possibility to configure limit used to retrieve objects for an API response added

#30 Updated by VVD VVD 11 months ago

Russian translation of:

setting_api_limit: Maximum number of items returned in a JSON/XML API response

is

setting_api_limit: Максимальное количество элементов, передаваемых в ответе JSON/XML API

#31 Updated by VVD VVD 10 months ago

What blocks this change?

#32 Updated by Txinto Vaz 9 months ago

+1

#33 Updated by VVD VVD 9 months ago

meineerde from IRC #redmine@freenode ask me to describe my use-case.
Sorry for my poor english, but I try to explain why this feature is important for me.

We are using Redmine for manage projects - tasks, issues, bugtracker. 10-20 active projects with hundreds active issues and 100+ active users - employees and customers. Employees are using "Spent time" in issues to record work time for every issue every day. The employees' salary depends on those records.
We have developed application on Java+Tomcat for create reports with values from "Spent time" fields. It using REST API to get values:

http://redmine.host/time_entries.json?spent_on=><BEGIN_OF_MONTH|END_OF_MONTH&offset=N&limit=10000

For example to get 20000 objects - 200 requests in ~3 minutes even on localhost. But it's faster with limit 10000 - ~20 seconds only.
After end of each month 10+ project managers (I'm one of them) using this application to create reports for each project. The employees salary calculated from this reports. Every mouth we have thousands of "Spent time" entries. It's too long for users wait several minutes for "simple" report, and even more - some browsers timedout connection during this time…

Please, commit this patch!

#34 Updated by vzvu 3k6k 6 months ago

Looks good. We've wrote a patch based on Domingo Galdos' one (Patch 0001) with Juno NISHIZAKI, sanak and me on Redmine patch meeting 4th.

The changes are as follows:
  • Added a test. (Patch 0002)
  • Moved the configuration value form from the "General" tab to the "API" tab and shorten the label name. (Patch 0003, 0004)
  • In the current implementation, if the limit parameter is less than 1, Redmine sets limit to 25. the limit is less than 25, the conditional statement is modified to be replaced by another one. (Patch 0005)

Go MAEDA wrote:

The user should not be able to set the value of API limit smaller than 100.
Probably some clients suppose that the value is always 100, smaller limit may affect compatibility with such clients.

Sounds reasonable, but currently the setting page doesn't have any validations. Do you have any good idea to apply the constraint?

#35 Updated by Mizuki ISHIKAWA 6 months ago

vzvu 3k6k wrote:

Go MAEDA wrote:

The user should not be able to set the value of API limit smaller than 100.
Probably some clients suppose that the value is always 100, smaller limit may affect compatibility with such clients.

Sounds reasonable, but currently the setting page doesn't have any validations. Do you have any good idea to apply the constraint?

The attached patch adds validation, though it may not be the best way to write it.

#36 Updated by Marius BALTEANU 6 months ago

  • Target version changed from Candidate for next major release to 4.2.0

The final result of the patch series posted by vzvu 3k6k and Mizuki looks good to me and all the tests pass: https://gitlab.com/redmine-org/redmine/-/pipelines/207877707

I propose to evaluate this for 4.2.0, but we should have Jean-Philippe agree on this because he rejected the patch some time ago. The feature is useful, for sure.

#37 Updated by Roberto Mendes 26 days ago

I need this. To get more than 100 items.

#38 Updated by Marius BALTEANU 20 days ago

  • Target version changed from 4.2.0 to 5.0.0

Also available in: Atom PDF