Defect #16239

Search does not return all the results

Added by Darth Vader almost 4 years ago. Updated about 2 years ago.

Status:ClosedStart date:
Priority:NormalDue date:
Assignee:-% Done:

0%

Category:Search engine
Target version:-
Resolution:Fixed Affected version:2.3.1

Description

Binami stach 2.3.1-0 on Windows 7 64 bit.
I was looking for issue #1028 and ran a text search for "Calibration" in the title only. The issue did not show up. It also said that there were 27 issues found but only 11 were displayed (see two search results screens).
First page of search results Second page of search results
When searching for "Calibration cfm", the issue showed up as part of the search. There were also 6 issues displayed and it said it found 6 issues. I also ran a filter for subject containing "calibration" and it returned 27 issues including the issue I was looking for.

Expected behaviour

I would expect the first search to find my document and to display all 27 of the documents found.

RedmineSearch1.PNG - First page of search results (27.8 KB) Darth Vader, 2014-03-05 19:58

RedmineSearch2.PNG - Second page of search results (4.11 KB) Darth Vader, 2014-03-05 19:58

search.diff Magnifier (703 Bytes) Darth Vader, 2014-04-07 18:24


Related issues

Related to Redmine - Feature #18631: Better search results pagination Closed
Duplicated by Redmine - Defect #15987: Wrong search pagination Closed
Duplicated by Redmine - Defect #8815: Invalid search results with paging Closed 2011-07-14

History

#1 Updated by Darth Vader almost 4 years ago

After investigating, it seems like the search displays 10 issues and the next page button displays issues based on date (as indicated by the last document date and the URL reflecting that date). You would expect it to just display the next 10 results in the search regardless of their creation date (like the filter does). It seems that the search has a number limit (to speed up the search) then sorts them by date. Subsequent searches use the last found issue date and run the search again. The search returns issues that are not sorted by date and newer issues could show up after the search limit and would get filtered out because the last issue on the previous page had an older time. To solve this problem, we sort the entire search result by date not just the limited search result:
We fixed it by commenting out line 77 in search_controller.rb

         r, c = s.singularize.camelcase.constantize.search(@tokens, projects_to_search,
           :all_words => @all_words,
           :titles_only => @titles_only,
# do not limit the number of search results as we need the full search set to sort properly         :limit => (limit+1),
           :offset => offset,
           :before => params[:previous].nil?)
         @results += r
         @results_by_type[s] += c
       end

#2 Updated by Darth Vader almost 4 years ago

I believe #15987 is a duplicate of this issue.

#3 Updated by Toshi MARUYAMA over 3 years ago

  • Category set to Search engine

#4 Updated by Toshi MARUYAMA over 3 years ago

#5 Updated by Daniel Felix over 3 years ago

This is really important for us. The search isn't usable for us if we search for items which results in 1000+ Results.

#6 Updated by Darth Vader over 3 years ago

This defect is still present in new versions of Redmine all the way up to 2.5.1. I have provided a solution and I am wondering why it is not being applied? Attached is a patch to make the implementation easier.

#7 Updated by Toshi MARUYAMA over 3 years ago

  • Duplicated by Defect #8815: Invalid search results with paging added

#8 Updated by Bill Houle over 3 years ago

Search pagination is just plain broken. The fundamental problem is that it hinges on the idea that no two 'objects' will have the same exact timestamp so that timestamp+ will always get the next page of results. As long as duplicate items land in the middle of the pagination block, the data will be displayed. But as soon as a set of duplicates appears on the modulo-10 boundary, you are left without full access to the results.

There are no guarantees that dups can be avoided. A heavily-trafficked system could have multiple users creating issues/wikis/etc with the same exact date. But even on a single-user system, the ability to bulk upload attachments, files, and DMSF docs will leave them all with the same identical date.

Darth's fix is not the full solution. There are 3 things that could be done:

1) Change the search pagination to use the same page size(s) as used for Issues display. Instead of hardcoding at 10, use the value in the Redmine Admin Settings to support multiple pagination limits. That way, the user can at least dynamically play with the boundaries upon which the search results will fall. What may not be displayed at mod-10 or mod-20 may become visible at mod-25. This fix may not be that difficult, and is perfectly inline with UI consistency.

2) Alter the upload function used in DMSF and for attachments/files/docs such that each file has a slight time offset rather than all in the batch being the same. (This will address the bulk import process, but does not solve for multi-user activity on a high-traffic site.)

3) The real solution: fundamentally change the way Search works. Treat a search as a complete record-set, cached by the client, and paginate over the fixed-in-time result rather than querying anew (with offset) for each next page.

#9 Updated by Jean-Philippe Lang about 3 years ago

#10 Updated by Dan Stoner about 2 years ago

I see that the related feature #18631 is closed, Resolution: Fixed.

Is there anything remaining on this Defect #16239?

#11 Updated by Darth Vader about 2 years ago

  • Status changed from New to Resolved

No. We can close this issue.

#12 Updated by Go MAEDA about 2 years ago

  • Status changed from Resolved to Closed

Thanks for pointing out. Closing.

#13 Updated by Go MAEDA about 2 years ago

  • Resolution set to Fixed

Also available in: Atom PDF