Project

General

Profile

Actions

Feature #1838

open

Bulk import for LDAP users

Added by paul k over 15 years ago. Updated about 7 years ago.

Status:
New
Priority:
Normal
Assignee:
-
Category:
Accounts / authentication
Target version:
-
Start date:
2008-09-01
Due date:
% Done:

0%

Estimated time:
Resolution:

Description

Hi,

if you configure LDAP for authentication, users are not visible unless they have at least logged in once. This is IMO not ideal in terms of usability. You have to tell your users to login but they can't do anything because there are not member of any project (yet) and you can't add them because they don't exist (chicken 'n egg, kinda). The patch implements a new 'import' action for the LDAP auth source. Warning: I'm pretty much a ruby noob so suggestions wrt style and functionality are always welcomed.

Cheers
Paul


Files

bulk_import_ldap.patch (5.93 KB) bulk_import_ldap.patch bulk import patch paul k, 2008-09-01 11:19
bulk_import_ldap2.patch (7.52 KB) bulk_import_ldap2.patch bulk import patch paul k, 2008-09-01 11:35
ldapimport.rb (2.57 KB) ldapimport.rb Thimios Dimopulos, 2009-11-10 20:52
import.php (2.11 KB) import.php Ritesh Sutaria, 2010-09-06 12:49
ldapimport (161 Bytes) ldapimport just bash script to run .rb. Also used for cron job rain man, 2010-09-20 16:36
ldapimport.rb (3.36 KB) ldapimport.rb file I modified rain man, 2010-09-20 16:36

Related issues

Has duplicate Redmine - Defect #24840: LDAP: Users who didn't login yet, are not accesible (and groups where no user has logged in yet aren't available as well) to assign to a projectClosed

Actions
Actions #1

Updated by paul k over 15 years ago

Grmpf, wrong patch. Please ignore the first one

Actions #2

Updated by Justin Grevich almost 15 years ago

Is there any interest in merging this into Redmine? I think it's a great feature. I would like to expand upon it in that the user is given a "bulk import" dialogue box which allows the admin to select which users to import.

Thanks

Actions #3

Updated by A G almost 15 years ago

Does not work for me. I receive an error when trying to access an LDAP entry.

Actions #4

Updated by Jens Goldhammer almost 15 years ago

+100!

Actions #5

Updated by Thomas D almost 15 years ago

A G wrote:

Does not work for me. I receive an error when trying to access an LDAP entry.

Same for me here. I don't know much about Ruby, so unfortunately I can't be of much help

Actions #6

Updated by Thimios Dimopulos over 14 years ago

We are also very interested in this feature, any chance that it reaches the trunk?

Actions #7

Updated by Thimios Dimopulos over 14 years ago

I didn't want to patch Redmine in order to get this feature, so based on the patch by paul k, i created the attached script that i call through a cron job using the runner script:

*/5 * * * * /var/www/RedmineInstallations/development/0.8-stable/script/runner /var/www/RedmineInstallations/indice/ldapimport.rb > /var/log/redmine/development/ldapimport.log 2>&1

I wrote a blog entry about it at: http://blog.indice.gr/2009/11/redmine-bulk-import-for-ldap-users.html

regards

Actions #8

Updated by Stanislav German-Evtushenko over 14 years ago

Greate work man! Thank you!
I made small improvement (login is always to lowcase)

--- ldapimport.rb.orig  2009-11-13 11:31:29.000000000 +0300
+++ ldapimport.rb       2009-11-13 11:42:42.000000000 +0300
@@ -21,7 +21,7 @@
                      :mail => AuthSourceLdap.get_attr(entry, self.attr_mail),
                      :auth_source_id => self.id ]
             #sanity checking (all the above attributes are required)
-            login = AuthSourceLdap.get_attr(entry, self.attr_login)
+            login = AuthSourceLdap.get_attr(entry, self.attr_login).downcase
             catch :SKIP do
               skip = false
               attrs.each { |e| 

Actions #9

Updated by Stanislav German-Evtushenko over 14 years ago

One else improvement is update email if changed.

--- ldapimport.rb.orig  2009-11-13 11:31:29.000000000 +0300
+++ ldapimport.rb       2009-11-13 13:36:18.000000000 +0300
@@ -37,7 +37,14 @@
               }
             end
             next if skip
-            if User.find(:first, :conditions => ["login=?", login])
+            if u = User.find(:first, :conditions => ["login=?", login])
+             # Update email if changed
+             if u.mail != attrs[0][:mail]
+               u.mail = attrs[0][:mail]
+               if u.save
+                  logger.debug("Email for user #{login} was updated") if logger
+               end
+             end
               logger.debug("User #{login} already there, skipping...") if logger
               skipped.push(login+'(exists)')
               next

Actions #10

Updated by Thimios Dimopulos over 14 years ago

Thanks for the improvements. In case you also need it, i added a method that removes users that are no longer present in the LDAP database:

def remove_non_existing_users
        ldap_con = initialize_ldap_con(self.account, self.account_password)
        found = 0
        logger.info("Removing users that do not exist in LDAP authentication source: #{self.name}.")
        @users = User.all
        @users.each { |user|

            search_filter = Net::LDAP::Filter.eq("sAMAccountName", user.login)
            entry = ldap_con.search(
                    :base => self.base_dn,
                    :filter => search_filter,
                    :attributes => ['dn', self.attr_firstname, self.attr_lastname, self.attr_mail, self.attr_login]
                )
            # ignore anonymous (empty login name and instance of AnonymousUser) and admin users
            if entry.empty?  && !user.login.empty? && user.class != AnonymousUser && user.login != "admin" 
                logger.info("The user with login name: #{user.login} does not exist any more in LDAP and will be removed")
                      found += 1
                User.delete(user.id)
            end
        }
        logger.info("Removed #{found} users that did not exist in LDAP authentication source: #{self.name}.")
end
Actions #11

Updated by Stanislav German-Evtushenko over 14 years ago

Thimios Dimopulos wrote:

Thanks for the improvements. In case you also need it, i added a method that removes users that are no longer present in the LDAP database:

[...]

Hm.. I think it's not so good thing, because:
1. Users are using in a history (I mean tasks, projects, wiki edit history, etc).
2. I have one special user (for example "admin") who aren't using LDAP. It used for administration and support (if LDAP DB have down).
3. If LDAP DB will accidentally be corrupted you can remove your active users.

Actions #12

Updated by Thimios Dimopulos over 14 years ago

You are absolutely right, deleting users just like that would lead to database corruption.

About the admin and anonymous users, i take that into account and do not manipulate them:

if entry.empty?  && !user.login.empty? && user.class != AnonymousUser && user.login != "admin" 

I updated the method so that it just locks users that do not exist in LDAP any more:

def lock_non_existing
        ldap_con = initialize_ldap_con(self.account, self.account_password)
        found = 0
        logger.info("Removing users that do not exist in LDAP authentication source: #{self.name}.")
        @users = User.all
        @users.each { |user| 

            search_filter = Net::LDAP::Filter.eq("sAMAccountName", user.login)
            entry = ldap_con.search(
                    :base => self.base_dn,
                    :filter => search_filter,
                    :attributes => ['dn', self.attr_firstname, self.attr_lastname, self.attr_mail, self.attr_login]
                )
            # ignore anonymous (empty login name and instance of AnonymousUser) and admin users 
            if entry.empty?  && !user.login.empty? && user.class != AnonymousUser && user.login != "admin" 
                logger.info("The user with login name: #{user.login} does not exist any more in LDAP and will be locked")
                      found += 1
                user.status=User::STATUS_LOCKED
                user.save
            end
        }
        logger.info("Locked #{found} users that did not exist in LDAP authentication source: #{self.name}.")
    end
Actions #13

Updated by Stanislav German-Evtushenko over 14 years ago

Thimios Dimopulos wrote:

You are absolutely right, deleting users just like that would lead to database corruption.

About the admin and anonymous users, i take that into account and do not manipulate them:
[...]

I updated the method so that it just locks users that do not exist in LDAP any more:

[...]

It's better way.
PS: LDAP doesn't have sAMAccountName. sAMAccountName is just Microsoft's thing.

Actions #14

Updated by Igor Kalashnikov over 14 years ago

Unable to import (undefined method `+' for nil:NilClass)

When I try to "bulk import"

Actions #15

Updated by Ritesh Sutaria over 13 years ago

PHP script to Bulk Import LDAP users in Redmine.

You may add in crontab to execute every 24 hours.

0 0 * * * * root /user/bin/php /path/phpscript >> /var/log/logfile

Actions #16

Updated by Stanislav German-Evtushenko over 13 years ago

Ritesh Sutaria wrote:

PHP script to Bulk Import LDAP users in Redmine.

You may add in crontab to execute every 24 hours.

0 0 * * * * root /user/bin/php /path/phpscript >> /var/log/logfile

This script works with PostgreSQL only. I think ldapimport.rb quite better because:
  • it uses own redmine engine for adding accounts
  • it isn't depend of database engine
Actions #17

Updated by Samu Tuomisto over 13 years ago

Stanislav German-Evtushenko wrote:

Ritesh Sutaria wrote:

PHP script to Bulk Import LDAP users in Redmine.

You may add in crontab to execute every 24 hours.

0 0 * * * * root /user/bin/php /path/phpscript >> /var/log/logfile

This script works with PostgreSQL only. I think ldapimport.rb quite better because:
  • it uses own redmine engine for adding accounts
  • it isn't depend of database engine

Seems to work ok also with redmine 1.01 - thanks!

I wonder why this user population feature is not in redmine core - I don't see any point using LDAP with redmine without populating user base and setting access rights for users before hand.

Actions #18

Updated by rain man over 13 years ago

Hi, I made some modifications to make this work in Active directory. It handled disabled accounts and additional logic to handle locking of accounts in redmine.

Actions #19

Updated by Samu Tuomisto over 13 years ago

rain man wrote:

Hi, I made some modifications to make this work in Active directory. It handled disabled accounts and additional logic to handle locking of accounts in redmine.

I tested the script but faced some errors. Could you advice me - how to solve following problem. Error messages from ldaplog.txt:

/opt/redmine-1.0.1-0/apps/redmine/script/../config/../vendor/rails/railties/lib/rails/gem_dependency.rb:119:Warning: Gem::Dependency#version_requirements is deprecated and will be removed on or after August 2010.  Use #requirement
/opt/redmine-1.0.1-0/apps/redmine/vendor/rails/railties/lib/commands/runner.rb:48: (eval):1: compile error (SyntaxError)
(eval):1: unknown regexp options - rd
(eval):1: no .<digit> floating literal anymore; put 0 before dot
/opt/redmine-1.0.1-0/apps/redmine/ldapimport.rb
                 ^
(eval):1: syntax error, unexpected tINTEGER
/opt/redmine-1.0.1-0/apps/redmine/ldapimport.rb
                  ^
        from /opt/redmine-1.0.1-0/ruby/lib/ruby/site_ruby/1.8/rubygems/custom_require.rb:31:in `eval'
        from /opt/redmine-1.0.1-0/apps/redmine/vendor/rails/railties/lib/commands/runner.rb:48
        from /opt/redmine-1.0.1-0/ruby/lib/ruby/site_ruby/1.8/rubygems/custom_require.rb:31:in `gem_original_require'
        from /opt/redmine-1.0.1-0/ruby/lib/ruby/site_ruby/1.8/rubygems/custom_require.rb:31:in `require'
        from /opt/redmine-1.0.1-0/apps/redmine/script/runner:3

My ruby on rails skills are unfortunately very limited.

Actions #20

Updated by Samu Tuomisto over 13 years ago

  • Status changed from New to Resolved

Samu Tuomisto wrote:

rain man wrote:

Hi, I made some modifications to make this work in Active directory. It handled disabled accounts and additional logic to handle locking of accounts in redmine.

I tested the script but faced some errors. Could you advice me - how to solve following problem. Error messages from ldaplog.txt:

[...]

My ruby on rails skills are unfortunately very limited.

This problem resolved - reasons were:
1) problem with gem mysql; solved by: http://oracleabc.com/b/archives/339
2) bitnami stack that I am not familiar with

Actions #21

Updated by Felix Schäfer over 13 years ago

  • Status changed from Resolved to New

Samu Tuomisto wrote:

This problem resolved

Yours yes, but the original request isn't resolved yet, reopening.

Actions #22

Updated by Anonymous over 13 years ago

Hi there,

I have tried the ldapimport.rb within a bash file with redmine-1.0.4 and i have got this error :

/services/redmine/script/ldapimport.rb:25:in `import': undefined method `include?' for nil:NilClass (NoMethodError)
        from /services/redmine-trunk/vendor/plugins/ruby-net-ldap-0.0.4/lib/net/ldap.rb:642:in `search'
        from /services/redmine-trunk/vendor/plugins/ruby-net-ldap-0.0.4/lib/net/ldap.rb:1175:in `search'
        from /services/redmine-trunk/vendor/plugins/ruby-net-ldap-0.0.4/lib/net/ldap.rb:1144:in `loop'
        from /services/redmine-trunk/vendor/plugins/ruby-net-ldap-0.0.4/lib/net/ldap.rb:1144:in `search'
        from /services/redmine-trunk/vendor/plugins/ruby-net-ldap-0.0.4/lib/net/ldap.rb:640:in `search'
        from /services/redmine/script/ldapimport.rb:15:in `import'
        from /services/redmine/script/ldapimport.rb:104
        from /usr/lib/ruby/site_ruby/1.8/rubygems/custom_require.rb:31:in `eval'
        from /usr/lib/ruby/gems/1.8/gems/rails-2.3.5/lib/commands/runner.rb:46
        from /usr/lib/ruby/site_ruby/1.8/rubygems/custom_require.rb:31:in `gem_original_require'
        from /usr/lib/ruby/site_ruby/1.8/rubygems/custom_require.rb:31:in `require'
        from /services/redmine/script/runner:3

My knowledge of Ruby language are very limited...

Thanks for any help,
Patrice

Actions #23

Updated by Alexandre Lissy almost 13 years ago

I've been able to run this ldapimport.rb script with success on my redmine 1.1.2 instance, only had to change the id of the AuthSourceLdap in find() at the first line, because our LDAP server is id=2.

Actions #24

Updated by Terence Mill over 12 years ago

Since redmine 1.2.1 there is a possibility to delete users, i think also exported to redmine api.
Would be nice to implement optional auto deletion this way.
Also there is a new plugin for ldap sync

Actions #25

Updated by yannick quenec'hdu over 12 years ago

Terence Mill wrote:

Since redmine 1.2.1 there is a possibility to delete users, i think also exported to redmine api.

How is possible to delete user in Rdmine and LDAP since 1.2.1 ?
I don't see this feature in roadmap. I tested with a redmine 1.2.1, if you delete a user in Redmine, this user isn't delete in LDAP.
If you delete user in LDAP, isn't delete in LDAP.

LDAP sync imposes a particular LDAP structure. It's not very generic.

Yannick

Actions #26

Updated by Terence Mill over 12 years ago

You can delete user in redmine via gui and i think via api too. I never said that the ldap plugin does sync that back to ldap.
For my understanding of ldap i would be not wished to sync application side user cache back to ldap more the other way round.
LDAP is central user service and the appluication (like redmine) will sync from ldap but never too.
Why don't u use plain redmine user base without ldap then?

Actions #27

Updated by Terence Mill over 12 years ago

You can delete user in redmine via gui and i think via api too. I never said that the ldap plugin does sync that back to ldap.
For my understanding of ldap it wouldn't be wished to sync application side user cache back to ldap more the other way round.
LDAP is central user service and the appluication (like redmine) will sync from ldap but never to ldap.
Why don't u use plain redmine user base without ldap then?

Actions #28

Updated by yannick quenec'hdu over 12 years ago

Terence Mill wrote:

You can delete user in redmine via gui and i think via api too. I never said that the ldap plugin does sync that back to ldap.

Because, this topics speak about LDAP sync. The feature "user delete" has existed for a long time, long before version 1.2.1.

For my understanding of ldap i would be not wished to sync application side user cache back to ldap more the other way round.
LDAP is central user service and the appluication (like redmine) will sync from ldap but never too.
Why don't u use plain redmine user base without ldap then?

I use a SSO with LDAP, We have centralized the creation of users with LDAP. When I delete an account, I delete in LDAP.
When I use Redmin, I use authentication with LDAP. Redmine work well to create a new account with LDAP authentication. But, when I delete user in my LDAP, This user is always present in Redmine. I think the feature (ldap authentication) is half-developed

Actions #29

Updated by Jérôme BATAILLE about 12 years ago

You can check :
https://github.com/Utopism/redmine_ldap_sync

a plugin that brings a few of these features.

Actions #30

Updated by Sebastian Paluch over 8 years ago

+1

Actions #31

Updated by Mischa The Evil about 7 years ago

  • Has duplicate Defect #24840: LDAP: Users who didn't login yet, are not accesible (and groups where no user has logged in yet aren't available as well) to assign to a project added
Actions #32

Updated by Svetly Minev about 7 years ago

Is this patch working under Redmine 3.3.2 / 3.3.X ???

Or is there any other solution how to import all AD users in Redmine?

I will appreciate every help

Thank you in advance

Actions

Also available in: Atom PDF