Security in Condor is a broad issue, with many aspects to consider. Because Condor's main purpose is to allow users to run arbitrary code on large numbers of computers, it is important to try to limit who can access a Condor pool and what privileges they have when using the pool. This section covers these topics.
There is a distinction between the kinds of resource attacks Condor can defeat, and the kinds of attacks Condor cannot defeat. Condor cannot prevent security breaches of users that can elevate their privilege to the root or administrator account. Condor does not run user jobs in sandboxes (standard universe jobs are a partial exception to this), so Condor cannot defeat all malicious actions by user jobs. An example of a malicious job is one that launches a distributed denial of service attack. Condor assumes that users are trustworthy. Condor can prevent unauthorized access to the Condor pool, to help ensure that only trusted users have access to the pool. In addition, Condor provides encryption and integrity checking, to ensure that data (both Condor's data and user jobs' data) has not been examined or tampered with.
Broadly speaking, the aspects of security in Condor may be categorized and described:
At the heart of Condor's security model is the notion that communications are subject to various security checks. A request from one Condor daemon to another may require authentication to prevent subversion of the system. A request from a user of Condor may need to be denied due to the confidential nature of the request. The security model handles these example situations and many more.
Requests to Condor are categorized into groups of access levels, based on the type of operation requested. The user of a specific request must be authorized at the required access level. For example, the executing the condor_ status command requires the READ access level. Actions that accomplish management tasks, such as shutting down or restarting of a daemon require an ADMINISTRATOR access level. See Section 3.6.8 for a full list of Condor's access levels and their meanings.
There are two sides to any communication or command invocation in Condor. One side is identified as the client, and the other side is identified as the daemon. The client is the party that initiates the command, and the daemon is the party that processes the command and responds. In some cases it is easy to distinguish the client from the daemon, while in other cases it is not as easy. Condor tools such as condor_ submit and condor_ config_val are clients. They send commands to daemons and act as clients in all their communications. For example, the condor_ submit command communicates with the condor_ schedd. Behind the scenes, Condor daemons also communicate with each other; in this case the daemon initiating the command plays the role of the client. For instance, the condor_ negotiator daemon acts as a client when contacting the condor_ schedd daemon to initiate matchmaking. Once a match has been found, the condor_ schedd daemon acts as a client and contacts the condor_ startd daemon.
Condor's security model is implemented using configuration. Commands in Condor are executed over TCP/IP network connections. While network communication enables Condor to manage resources that are distributed across an organization (or beyond), it also brings in security challenges. Condor must have ways of ensuring that commands are being sent by trustworthy users. Jobs that are operating on sensitive data must be allowed to use encryption such that the data is not seen by outsiders. Jobs may need assurance that data has not been tampered with. These issues can be addressed with Condor's authentication, encryption, and integrity features.
Because of the wide range of environments and security demands necessary, Condor must be flexible. Configuration provides this flexibility. The process by which Condor determines the security settings that will be used when a connection is established is called security negotiation. Security negotiation's primary purpose is to determine which of the features of authentication, encryption, and integrity checking will be enabled for a connection. In addition, since Condor supports multiple technologies for authentication and encryption, security negotiation also determines which technology is chosen for the connection.
Security negotiation is a completely separate process from matchmaking, and should not be confused with any specific function of the condor_ negotiator daemon.
The configuration macro names that determine what features will be used during client-daemon communication follow the pattern:
portion of the macro name determines which security feature's
policy is being set.
<feature> may be any one of
AUTHENTICATION ENCRYPTION INTEGRITY NEGOTIATION
<context> component of the security policy macros can be
used to craft a fine-grained security policy based on the type of
communication taking place.
<context> may be any one of
CLIENT READ WRITE ADMINISTRATOR CONFIG OWNER DAEMON NEGOTIATOR DEFAULT
Any of these constructed configuration macros may be set to any of the following values:
REQUIRED PREFERRED OPTIONAL NEVER
Security negotiation resolves various client-daemon combinations of desired security features in order to set a policy.
As an example, consider Frida the scientist. Frida wants to avoid authentication when possible. She sets
SEC_DEFAULT_AUTHENTICATION = OPTIONALThe machine running the condor_ schedd to which Frida will remotely submit jobs, however, is operated by a security-conscious system administrator who dutifully sets:
SEC_DEFAULT_AUTHENTICATION = REQUIREDWhen Frida submits her jobs, Condor's security negotiation determines that authentication will be used, and allows the command to continue.
Whether or not security negotiation occurs depends on the setting at both the client and daemon side of the configuration variable(s) defined by SEC_*_NEGOTIATION. SEC_DEFAULT_NEGOTIATION is a variable representing the entire set of configuration variables for NEGOTIATION. For the client side setting, the only definitions that make sense are REQUIRED and NEVER. For the daemon side setting, the PREFERRED value makes no sense. Table 3.1 shows how security negotiation resolves various client-daemon combinations of security negotiation policy settings. Within the table, Yes means the security negotiation will take place. No means it will not. Fail means that the policy settings are incompatible and the communication cannot continue.
Enabling authentication, encryption, and integrity checks is dependent on security negotiation taking place. The enabled security negotiation further sets the policy for these other features. Table 3.2 shows how security features are resolved for client-daemon combinations of security feature policy settings. Like Table 3.1, Yes means the feature will be utilized. No means it will not. Fail implies incompatibility and the feature cannot be resolved.
The enabling of encryption and/or integrity checks is dependent on authentication taking place. The authentication provides a key exchange. The key is needed for both encryption and integrity checks.
SEC_CLIENT_<feature> determines the policy for all
The policy for incoming commands
(the daemon side of the communication)
takes a more fine-grained approach that implements a set of
access levels for the received command.
For example, it is desirable to have all incoming administrative
requests require authentication.
Inquiries on pool status may not be so restrictive.
To implement this, the administrator configures the policy:
SEC_ADMINISTRATOR_AUTHENTICATION = REQUIRED SEC_READ_AUTHENTICATION = OPTIONAL
DEFAULT value for
<context> provides a way
to set a policy for all access levels
that do not have a specific configuration variable defined.
Authentication and encryption can each be accomplished by a variety of methods or technologies. Which method is utilized is determined during security negotiation.
The configuration macros that determine the methods to use for authentication and/or encryption are
These macros are defined by a comma or space delimited list of possible methods to use. Section 3.6.3 lists all implemented authentication methods. Section 3.6.5 lists all implemented encryption methods.
The client side of any communication uses one of two macros to specify whether authentication is to occur:
For the daemon side, there are seven macros to specify whether authentication is to take place, based upon the necessary access level:
SEC_DEFAULT_AUTHENTICATION SEC_READ_AUTHENTICATION SEC_WRITE_AUTHENTICATION SEC_ADMINISTRATOR_AUTHENTICATION SEC_CONFIG_AUTHENTICATION SEC_OWNER_AUTHENTICATION SEC_DAEMON_AUTHENTICATION SEC_NEGOTIATOR_AUTHENTICATION
As an example, the macro defined in the configuration file for a daemon as
SEC_WRITE_AUTHENTICATION = REQUIREDsignifies that the daemon must authenticate the client for any communication that requires the WRITE access level. If the daemon's configuration contains
SEC_DEFAULT_AUTHENTICATION = REQUIREDand does not contain any other security configuration for
AUTHENTICATION, then this default defines the daemon's needs for authentication over all access levels. Where a specific macro is defined, the more specific value takes precedence over the default definition.
If authentication is to be done, then the communicating parties must negotiate a mutually acceptable method of authentication to be used. A list of acceptable methods may be provided by the client, using the macros
SEC_DEFAULT_AUTHENTICATION_METHODS SEC_CLIENT_AUTHENTICATION_METHODSA list of acceptable methods may be provided by the daemon, using the macros
SEC_DEFAULT_AUTHENTICATION_METHODS SEC_READ_AUTHENTICATION_METHODS SEC_WRITE_AUTHENTICATION_METHODS SEC_ADMINISTRATOR_AUTHENTICATION_METHODS SEC_CONFIG_AUTHENTICATION_METHODS SEC_OWNER_AUTHENTICATION_METHODS SEC_DAEMON_AUTHENTICATION_METHODS SEC_NEGOTIATOR_AUTHENTICATION_METHODSThe methods are given as a comma-separated list of acceptable values. These variables list the authentication methods that are available to be used. The ordering of the list defines preference; the first item in the list indicates the highest preference. Defined values are
GSI SSL KERBEROS PASSWORD FS FS_REMOTE NTSSPI CLAIMTOBE ANONYMOUS
For example, a client may be configured with:
SEC_CLIENT_AUTHENTICATION_METHODS = FS, GSIand a daemon the client is trying to contact with:
SEC_DEFAULT_AUTHENTICATION_METHODS = GSI
Security negotiation will determine that GSI authentication is the only compatible choice. If there are multiple compatible authentication methods, security negotiation will make a list of acceptable methods and they will be tried in order until one succeeds.
As another example, the macro
SEC_DEFAULT_AUTHENTICATION_METHODS = KERBEROS, NTSSPIindicates that either Kerberos or Windows authentication may be used, but Kerberos is preferred over Windows. Note that if the client and daemon agree that multiple authentication methods may be used, then they are tried in turn. For instance, if they both agree that Kerberos or NTSSPI may be used, then Kerberos will be tried first, and if there is a failure for any reason, then NTSSPI will be tried.
If the configuration for a machine does not define any variable
then Condor uses a default value of
Authentication will be required for
any operation which modifies the job queue,
such as condor_ qedit and condor_ rm.
If the configuration for a machine does not define any variable
the default value for a Unix machine is
This default value for a Windows machine is
A simple introduction to this type of authentication defines Condor's use of terminology, and it illuminates the needed items that Condor must access to do this authentication. Assume that A authenticates to B. In this example, A is the client, and B is the daemon within their communication. This example's one-way authentication implies that B is verifying the identity of A, using the certificate A provides, and utilizing B's own set of trusted CAs (Certification Authorities). Client A provides its certificate (or proxy) to daemon B. B does two things: B checks that the certificate is valid, and B checks to see that the CA that signed A's certificate is one that B trusts.
For the GSI authentication protocol, an X.509 certificate is required. Files with predetermined names hold a certificate, a key, and optionally, a proxy. A separate directory has one or more files that become the list of trusted CAs.
Allowing Condor to do this GSI authentication requires knowledge of the locations of the client A's certificate and the daemon B's list of trusted CAs. When one side of the communication (as either client A or daemon B) is a Condor daemon, these locations are determined by configuration or by default locations. When one side of the communication (as a client A) is a user of Condor (the process owner of a Condor tool, for example condor_ submit), these locations are determined by the pre-set values of environment variables or by default locations.
For a Condor daemon, the certificate may be a single host certificate, and all Condor daemons on the same machine may share the same certificate. In some cases, the certificate can also be copied to other machines, where local copies are necessary. This may occur only in cases where a single host certificate can match multiple host names, something that is beyond the scope of this manual. The certificates must be protected by access rights to files, since the password file is not encrypted.
The specification of the location of the necessary files through configuration uses the following precedence.
GSI_DAEMON_CERT = $(GSI_DAEMON_DIRECTORY)/hostcert.pem GSI_DAEMON_KEY = $(GSI_DAEMON_DIRECTORY)/hostkey.pem GSI_DAEMON_TRUSTED_CA_DIR = $(GSI_DAEMON_DIRECTORY)/certificates
Note that no proxy is assumed in this case.
Here is an example portion of the configuration file that would enable and require GSI authentication, along with a minimal set of other variables to make it work. Note that the last entry (GSI_DAEMON_NAME) in this example must be on a single line; this example is broken onto two lines for formatting reasons.
SEC_DEFAULT_AUTHENTICATION = REQUIRED SEC_DEFAULT_AUTHENTICATION_METHODS = GSI GSI_DAEMON_DIRECTORY = /path/to/daemon/credential.location GSI_DAEMON_NAME = /C=US/O=Condor/O=University of Wisconsin /OU=Computer Sciences Department/CNemail@example.com
The SEC_DEFAULT_AUTHENTICATION macro specifies that authentication is required for all communications. This single macro covers all communications, but could be replaced with a set of macros that require authentication for only specific communications.
The macro GSI_DAEMON_DIRECTORY is specified to give Condor a single place to find the daemon's certificate. This path may be a directory on a shared file system such as AFS. Alternatively, this path name can point to local copies of the certificate stored in a local file system.
When a daemon acts as the client within authentication, the daemon needs a listing of those from which it will accept certificates.
The macro GSI_DAEMON_NAME configuration macro provides daemons with a distinguished name to use for X.509 authentication. This name is specified with the following format
GSI_DAEMON_NAME = /C=?/O=?/O=?/OU=?/CN=<daemon_name@domain>A complete example that has the question marks filled in and the daemon's user name filled in is given in the example configuration above.
Condor will also need a way to map an X.509 distinguished
name to a Condor user id.
There are two ways to accomplish this mapping.
For a first way to specify the mapping, see
to use Condor's unified map file.
The second way to do the mapping is within
an administrator-maintained GSI-specific file called an X.509 map file,
mapping from X509 Distinguished Name (DN) to Condor user id.
It is similar to a Globus grid map file except that it is only used for
mapping to a user id, not for authorization.
Information about authorization can be found in
Entries (lines) in the file each contain two items.
The first item in an entry is the
X.509 certificate subject name, and it is enclosed in quotes
(using the character
The second item is the Condor user id.
The two items in an entry are separated by tab or space character(s).
Here is an example of an entry in an X.509 map file.
Entries must be on a single line; this example is broken
onto two lines for formatting reasons.
"/C=US/O=Globus/O=University of Wisconsin/ OU=Computer Sciences Department/CN=Alice Smith" asmith
Condor finds the map file in one of three ways. If the configuration variable GRIDMAP is defined, it gives the full path name to the map file. When not defined, Condor looks for the map file in
$(GSI_DAEMON_DIRECTORY)/grid-mapfileIf GSI_DAEMON_DIRECTORY is not defined, then the third place Condor looks for the map file is given by
The user specifies the location of a certificate, proxy, etc. in one of two ways:
X509_USER_PROXY gives the path and file name of the proxy. This proxy will have been created using the grid-proxy-init program, which will place the proxy in the /tmp directory with the file name being determined by the format:
/tmp/x509up_uXXXXThe specific file name is given by substituting the
XXXXcharacters with the UID of the user. Note that when a valid proxy is used, the certificate and key locations are not needed.
X509_USER_CERT gives the path and file name of the certificate. It is also used if a proxy location has been checked, but the proxy is no longer valid.
X509_USER_KEY gives the path and file name of the key. Note that most keys are password encrypted, such that knowing the location could not lead to using the key.
X509_CERT_DIR gives the path to the directory containing the list of trusted CAs.
All SSL authentication is mutual authentication in Condor. This means that when SSL authentication is used and when one process communicates with another, each process must be able to verify the signature on the certificate presented by the other process. The process that initiates the connection is the client, and the process that receives the connection is the server. For example, when a condor_ startd daemon authenticates with a condor_ collector daemon to provide a machine ClassAd, the condor_ startd daemon initiates the connection and acts as the client, and the condor_ collector daemon acts as the server.
The names and locations of keys and certificates for clients, servers, and the files used to specify trusted certificate authorities (CAs) are defined by settings in the configuration files. The contents of the files are identical in format and interpretation to those used by other systems which use SSL, such as Apache httpd.
The configuration variables AUTH_SSL_CLIENT_CERTFILE and AUTH_SSL_SERVER_CERTFILE specify the file location for the certificate file for the initiator and recipient of connections, respectively. Similarly, the configuration variables AUTH_SSL_CLIENT_KEYFILE and AUTH_SSL_SERVER_KEYFILE specify the locations for keys.
The configuration variables AUTH_SSL_SERVER_CAFILE and AUTH_SSL_CLIENT_CAFILE each specify a path and file name, providing the location of a file containing one or more certificates issued by trusted certificate authorities. Similarly, AUTH_SSL_SERVER_CADIR and AUTH_SSL_CLIENT_CADIR each specify a directory with one or more files, each which may contain a single CA certificate. The directories must be prepared using the OpenSSL c_rehash utility.
If Kerberos is used for authentication, then a mapping from a Kerberos domain (called a realm) to a Condor UID domain is necessary. There are two ways to accomplish this mapping. For a first way to specify the mapping, see section 3.6.4 to use Condor's unified map file. A second way to specify the mapping defines the configuration variable KERBEROS_MAP_FILE to define a path to an administrator-maintained Kerberos-specific map file. The configuration syntax is
KERBEROS_MAP_FILE = /path/to/etc/condor.kmap
Lines within this map file have the syntax
KERB.REALM = UID.domain.name
Here are two lines from a map file to use as an example:
CS.WISC.EDU = cs.wisc.edu ENGR.WISC.EDU = ee.wisc.edu
If a KERBEROS_MAP_FILE configuration variable is defined and set, then all permitted realms must be explicitly mapped. If no map file is specified, then Condor assumes that the Kerberos realm is the same as the Condor UID domain.
The configuration variable CONDOR_SERVER_PRINCIPAL defines the name of a Kerberos principal. If CONDOR_SERVER_PRINCIPAL is not defined, then the default value used is "host". A principal specifies a unique name to which a set of credentials may be assigned.
Condor takes the specified (or default) principal and appends a slash character, the host name, an '@' (at sign character), and the Kerberos realm. As an example, the configuration
CONDOR_SERVER_PRINCIPAL = condor-daemonresults in Condor's use of
condor-daemon/the.host.name@YOUR.KERB.REALMas the server principal.
Here is an example of configuration settings that use Kerberos for authentication and require authentication of all communications of the write or administrator access level.
SEC_WRITE_AUTHENTICATION = REQUIRED SEC_WRITE_AUTHENTICATION_METHODS = KERBEROS SEC_ADMINISTRATOR_AUTHENTICATION = REQUIRED SEC_ADMINISTRATOR_AUTHENTICATION_METHODS = KERBEROS
Kerberos authentication on Unix platforms requires access to various files that usually are only accessible by the root user. At this time, the only supported way to use KERBEROS authentication on Unix platforms is to start daemons Condor as user root.
The password method provides mutual authentication through the use of a shared secret. This is often a good choice when strong security is desired, but an existing Kerberos or X.509 infrastructure is not in place. Password authentication is available on both Unix and Windows. It currently can only be used for daemon-to-daemon authentication. The shared secret in this context is referred to as the pool password.
Before a daemon can use password authentication, the pool password must be stored on the daemon's local machine. On Unix, the password will be placed in a file defined by the configuration variable SEC_PASSWORD_FILE . This file will be accessible only by the UID that Condor is started as. On Windows, the same secure password store that is used for user passwords will be used for the pool password (see section 6.2.3).
Storing the pool password is done via the -c option to condor_ store_cred. Running
condor_store_cred -c addwill prompt for the pool password and store it on the local machine, making it available for daemons to use for authentication. The condor_ master must be running for this command to work.
In addition, storing the pool password to a given machine requires
CONFIG-level access. For example, if the pool password should
only be set locally, and only by root, the following would be placed in
the global configuration file.
ALLOW_CONFIG = root@mydomain/$(IP_ADDRESS)
It is also possible to set the pool password remotely, but this is
recommended only if it can be done over an encrypted channel. This is
possible on Windows, for example, in an environment where common
accounts exist across all the machines in the pool. In this case,
ALLOW_CONFIG can be set to allow the Condor administrator (who
in this example has an account condor common to all machines in
the pool) to set the password from the central manager as follows.
ALLOW_CONFIG = condor@mydomain/$(CONDOR_HOST)The Condor administrator then executes
condor_store_cred -c -n host.mydomain addfrom the central manager to store the password to a given machine. Since the condor account exists on both the central manager and
host.mydomain, the NTSSPI authentication method can be used to authenticate and encrypt the connection. condor_ store_cred will warn and prompt for cancellation, if the channel is not encrypted for whatever reason (typically because common accounts do not exist or Condor's security is misconfigured).
When a daemon is authenticated using a pool password, its security
$(UID_DOMAIN) is taken from the daemon's configuration. The
variables for authorization
should restrict access using this name. For example,
ALLOW_DAEMON = condor_pool@mydomain/*, condor@mydomain/$(IP_ADDRESS) ALLOW_NEGOTIATOR = condor_pool@mydomain/$(CONDOR_HOST)This configuration allows remote
NEGOTIATOR-level access, if the pool password is known. Local daemons authenticated as
condor@mydomainare also allowed access. This is done so local authentication can be done using another method such as
This form of authentication utilizes the ownership of a file in the identity verification of a client. A daemon authenticating a client requires the client to write a file in a specific location (/tmp). The daemon then checks the ownership of the file. The file's ownership verifies the identity of the client. In this way, the file system becomes the trusted authority. This authentication method is only appropriate for clients and daemons that are on the same computer.
Condor's unified map file allows the mappings from authenticated names to a Condor canonical user name to be specified as a single list within a single file. The location of the unified map file is defined by the configuration variable CERTIFICATE_MAPFILE ; it specifies the path and file name of the unified map file. Each mapping is on its own line of the unified map file. Each line contains 3 fields, separated by white space (space or tab characters):
Allowable authentication method names are the same as used to define any of the configuration variables SEC_*_AUTHENTICATION_METHODS, as repeated here:
GSI SSL KERBEROS PASSWORD FS FS_REMOTE NTSSPI CLAIMTOBE ANONYMOUS
The fields that represent an authenticated name and the canonical Condor user name may utilize regular expressions as defined by PCRE (Perl-Compatible Regular Expressions). Due to this, more than one line (mapping) within the unified map file may match. Lookups are therefore defined to use the first mapping that matches.
The default map file contains a mapping that matches for each authentication method that Condor implements. The unusual string for the GSI authentication entry instructs Condor to use GSI-specific way of locating the needed GSI map file, as shown in section 3.6.3. The default map file:
FS (.*) \1 FS_REMOTE (.*) \1 GSI (.*) GSS_ASSIST_GRIDMAP SSL (.*) \1 KERBEROS (.*) \1 NTSSPI (.*) \1 CLAIMTOBE (.*) \1 PASSWORD (.*) \1
The client uses one of two macros to enable or disable encryption:
For the daemon, there are seven macros to enable or disable encryption:
SEC_DEFAULT_ENCRYPTION SEC_READ_ENCRYPTION SEC_WRITE_ENCRYPTION SEC_ADMINISTRATOR_ENCRYPTION SEC_CONFIG_ENCRYPTION SEC_OWNER_ENCRYPTION SEC_DAEMON_ENCRYPTION SEC_NEGOTIATOR_ENCRYPTION
As an example, the macro defined in the configuration file for a daemon as
SEC_CONFIG_ENCRYPTION = REQUIREDsignifies that any communication that changes a daemon's configuration must be encrypted. If a daemon's configuration contains
SEC_DEFAULT_ENCRYPTION = REQUIREDand does not contain any other security configuration for ENCRYPTION, then this default defines the daemon's needs for encryption over all access levels. Where a specific macro is present, its value takes precedence over any default given.
If encryption is to be done, then the communicating parties must find (negotiate) a mutually acceptable method of encryption to be used. A list of acceptable methods may be provided by the client, using the macros
SEC_DEFAULT_CRYPTO_METHODS SEC_CLIENT_CRYPTO_METHODSA list of acceptable methods may be provided by the daemon, using the macros
SEC_DEFAULT_CRYPTO_METHODS SEC_READ_CRYPTO_METHODS SEC_WRITE_CRYPTO_METHODS SEC_ADMINISTRATOR_CRYPTO_METHODS SEC_CONFIG_CRYPTO_METHODS SEC_OWNER_CRYPTO_METHODS SEC_DAEMON_CRYPTO_METHODS SEC_NEGOTIATOR_CRYPTO_METHODS
The methods are given as a comma-separated list of acceptable values. These variables list the encryption methods that are available to be used. The ordering of the list gives preference; the first item in the list indicates the highest preference. Possible values are
An integrity check assures that the messages between communicating parties have not been tampered with. Any change, such as addition, modification, or deletion can be detected. Through configuration macros, both the client and the daemon can specify whether an integrity check is required of further communication.
The client uses one of two macros to enable or disable an integrity check:
For the daemon, there are seven macros to enable or disable an integrity check:
SEC_DEFAULT_INTEGRITY SEC_READ_INTEGRITY SEC_WRITE_INTEGRITY SEC_ADMINISTRATOR_INTEGRITY SEC_CONFIG_INTEGRITY SEC_OWNER_INTEGRITY SEC_NEGOTIATOR_INTEGRITY
As an example, the macro defined in the configuration file for a daemon as
SEC_CONFIG_INTEGRITY = REQUIREDsignifies that any communication that changes a daemon's configuration must have its integrity assured. If a daemon's configuration contains
SEC_DEFAULT_INTEGRITY = REQUIREDand does not contain any other security configuration for
INTEGRITY, then this default defines the daemon's needs for integrity checks over all access levels. Where a specific macro is present, its value takes precedence over any default given.
A signed MD5 checksum is currently the only available method for integrity checking. Its use is implied whenever integrity checks occur. If more methods are implemented, then there will be further macros to allow both the client and the daemon to specify which methods are acceptable.
A configuration file is provided when Condor is installed. No security features are enabled within the configuration as distributed. Included as comments within the configuration file is an example suggesting settings that enable security features. Here is that example of the daemon-side portion.
SEC_DEFAULT_NEGOTIATION = REQUIRED SEC_DEFAULT_AUTHENTICATION = REQUIRED SEC_DEFAULT_ENCRYPTION = REQUIRED SEC_DEFAULT_INTEGRITY = REQUIRED SEC_DEFAULT_AUTHENTICATION_METHODS = KERBEROS, FS SEC_DEFAULT_CRYPTO_METHODS = 3DES, BLOWFISH
This set of configuration macros forces security negotiation to occur, and sets the features to be used at all times. All communication is authenticated with Kerberos, unless the client does not use Kerberos, but supports File System (FS) authentication, in which case FS authentication is used. All communication is both encrypted and integrity checked to make sure that messages are not modified or corrupted. The encryption is preferably with triple DES, but Blowfish will be used if the client does not use 3DES, but does use Blowfish.
Authorization protects resource usage by granting or denying access requests made to the resources. It defines who is allowed to do what.
Unlike authentication, encryption, and integrity checks, which can be configured by both client and server, authorization is used only by a server. The authorization portion of the security of a Condor pool is based on a set of configuration macros. The macros list which user/daemon will be authorized to issue what request given a specific access level.
These configuration macros define a set of users that will be allowed to (or denied from) carrying out various Condor commands. Each access level may have its own list of authorized users. A complete list of the authorization macros:
ALLOW_READ ALLOW_WRITE ALLOW_ADMINISTRATOR ALLOW_CONFIG ALLOW_OWNER ALLOW_NEGOTIATOR DENY_READ DENY_WRITE DENY_ADMINISTRATOR DENY_CONFIG DENY_OWNER DENY_NEGOTIATOR
Each macro is defined by a comma-separated list of fully qualified users. Each fully qualified user is described using the following format:
username@domain/hostnameThe information to the left of the slash character describes a user within a domain. The information to the right of the slash character describes one or more machines from which the user would be issuing a command. This host name may take the form of either a fully qualified host name of the form
bird.cs.wisc.eduor an IP address of the form
An example is
Within the format, wild card characters (the asterisk, *) are allowed. The use of wild cards is limited to one wild card on either side of the slash character. A wild card character used in the host name is further limited to come at the beginning of a fully qualified host name or at the end of an IP address. For example,
*@cs.wisc.edu/bird.cs.wisc.edurefers to any user that comes from
cs.wisc.edu, where the command is originating from the machine
bird.cs.wisc.edu. Another valid example,
firstname.lastname@example.org/*.cs.wisc.edurefers to commands coming from any machine within the
cs.wisc.edudomain, and issued by
zmiller. A third valid example,
*@cs.wisc.edu/*refers to commands coming from any user within the
cs.wisc.edudomain where the command is issued from any machine. A fourth valid example,
*@cs.wisc.edu/128.105.*refers to commands coming from any user within the
cs.wisc.edudomain where the command is issued from machines within the network that match the first two octets of the IP address.
If the set of machines is specified by an IP address, then further specification using a net mask identifies a physical set (subnet) of machines. This physical set of machines is specified using the form
networkis an IP address. The net mask takes one of two forms. It may be a decimal number which refers to the number of leading bits of the IP address that are used in describing a subnet. Or, the net mask may take the form of
dare decimal numbers that each specify an 8-bit mask. An example net mask is
255.255.192.0which specifies the bit mask
A single complete example of a configuration variable that uses a net mask is
ALLOW_WRITE = email@example.com/18.104.22.168/17User
cs.wisc.edudomain is given write authorization when originating from machines that match their leftmost 17 bits of the IP address.
This flexible set of configuration macros could used to define conflicting authorization. Therefore, the following protocol defines the precedence of the configuration macros.
An example of the configuration variables for the user-side authorization is derived from the necessary access levels as described in Section 3.6.1.
ALLOW_READ = *@cs.wisc.edu/* ALLOW_WRITE = *@cs.wisc.edu/*.cs.wisc.edu ALLOW_ADMINISTRATOR = firstname.lastname@example.org/*.cs.wisc.edu ALLOW_CONFIG = email@example.com/*.cs.wisc.edu ALLOW_NEGOTIATOR = firstname.lastname@example.org/$(COLLECTOR_HOST)
This example configuration presumes that the condor_ collector and condor_ negotiator daemons are running on the same machine.
This example configuration authorizes
any user in the
cs.wisc.edu domain to
carry out a request that requires the
READ access level
from any machine.
Any user in the
cs.wisc.edu domain may
carry out a request that requires the
WRITE access level
from any machine in the
Only the user called
carry out a request that requires the
ADMINISTRATOR access level
from any machine in the
The administrator, logged into any machine within
cs.wisc.edu domain is authorized at the
CONFIG access level.
Only the negotiator daemon, running as
condor on the machine defined by the
NEGOTIATOR_HOST macro is authorized
NEGOTIATOR access level.
And, the last line of the example presumes that there is a
user called condor, and that the daemons have all been started
up as this user.
In the local configuration file for each host, the host's owner should be authorized as the owner of the machine. An example of the entry in the local configuration file:
ALLOW_OWNER = email@example.com/hostname.cs.wisc.eduIn this example the owner has a login of
username, and the machine's name is represented by
To set up and configure secure communications in Condor, authentication, encryption, and integrity checks can be used. However, these come at a cost: performing strong authentication can take a significant amount of time, and generating the cryptographic keys for encryption and integrity checks can take a significant amount of processing power.
The Condor system makes many network connections between different daemons. If each one of these was to be authenticated, and new keys were generated for each connection, Condor would not be able to scale well. Therefore, Condor uses the concept of sessions to cache relevant security information for future use and greatly speed up the establishment of secure communications between the various Condor daemons.
A new session is established the first time a connection is made from one daemon to another. Each session has a fixed lifetime after which it will expire and a new session will need to be created again. But while a valid session exists, it can be re-used as many times as needed, thereby preventing the need to continuously re-establish secure connections. Each entity of a connection will have access to a session key that proves the identity of the other entity on the opposing side of the connection. This session key is exchanged securely using a strong authentication method, such as Kerberos or GSI. Other authentication methods, such as NTSSPI, FS_REMOTE, CLAIMTOBE, and ANONYMOUS, do not support secure key exchange. An entity listening on the wire may be able to impersonate the client or server in a session that does not use a strong authentication method.
Establishing a secure session requires that either the encryption or the integrity options be enabled. If the encryption capability is enabled, then the session will be restarted using the session key as the encryption key. If integrity capability is enabled, then the checksum includes the session key even though it is not transmitted. Without either of these two methods enabled, it is possible for an attacker to use an open session to make a connection to a daemon and use that connection for nefarious purposes. It is strongly recommended that if you have authentication turned on, you should also turn on integrity and/or encryption.
The configuration parameter SEC_DEFAULT_NEGOTIATION will allow
a user to set the default level of secure sessions in Condor.
Like other security settings, the possible values for this parameter can be
If you disable sessions and you have authentication turned
on, then most authentication (other than commands like
condor_ submit) will fail because Condor requires sessions when you
have security turned on.
On the other hand, if you are not using strong security in Condor, but
you are relying on the default host-based security, turning off
sessions may be useful in certain situations. These might include debugging problems
with the security session management or slightly decreasing the memory
consumption of the daemons, which keep track of the sessions in use.
Session lifetimes for specific daemons are already properly configured in the default installation of Condor. Condor tools such as condor_ q and condor_ status create a session that expires after one minute. Theoretically they should not create a session at all, because the session cannot be reused between program invocations, but this is difficult to do in the general case. This allows a very small window of time for any possible attack, and it helps keep the memory footprint of running daemons down, because they are not keeping track of all of the sessions. The session durations may be manually tuned by using macros in the configuration file, but this is not recommended.
This section describes the mechanisms for setting up Condor's host-based security. This is now an outdated form of implementing security levels for machine access. It remains available and documented for purposes of backward compatibility. If used at the same time as the user-based authorization, the two specifications are merged together.
The host-based security paradigm allows control over which machines can join a Condor pool, which machines can find out information about your pool, and which machines within a pool can perform administrative commands. By default, Condor is configured to allow anyone to view or join a pool. It is recommended that this parameter is changed to only allow access from machines that you trust.
This section discusses how the host-based security works inside Condor. It lists the different levels of access and what parts of Condor use which levels. There is a description of how to configure a pool to grant or deny certain levels of access to various machines. Configuration examples and the settings of configuration variables using the condor_ config_val command complete this section.
Inside the Condor daemons or tools that use DaemonCore (see section 3.9 for details), most tasks are accomplished by sending commands to another Condor daemon. These commands are represented by an integer value to specify which command is being requested, followed by any optional information that the protocol requires at that point (such as a ClassAd, capability string, etc). When the daemons start up, they will register which commands they are willing to accept, what to do with arriving commands, and the access level required for each command. When a command request is received by a daemon, Condor identifies the access level required and checks the IP address of the sender to verify that it satisfies the allow/deny settings from the configuration file. If permission is granted, the command request is honored; otherwise, the request will be aborted.
Settings for the access levels in the global configuration file will affect all the machines in the pool. Settings in a local configuration file will only affect the specific machine. The settings for a given machine determine what other hosts can send commands to that machine. If a machine foo is to be given administrator access on machine bar, place foo in bar's configuration file access list (not the other way around).
The following are the various access levels that commands within Condor can be registered with:
IMPORTANT: For a machine to join a Condor pool, the machine must have both WRITE permission AND READ permission. WRITE permission is not enough.
IMPORTANT: Giving ADMINISTRATOR privileges to a machine grants administrator access for the pool to ANY USER on that machine. This includes any users who can run Condor jobs on that machine. It is recommended that ADMINISTRATOR access is granted with due diligence.
Starting with version 6.3.2, Condor provides a mechanism for more fine-grained control over the configuration settings that can be modified remotely with condor_ config_val.
Host-based security access permissions are specified in configuration files.
ADMINISTRATOR and NEGOTIATOR access default to the central manager machine. OWNER access defaults to the local machine, as well as any machines given with ADMINISTRATOR access. CONFIG access is not granted to any machine as its default. These defaults are sufficient for most pools, and should not be changed without a compelling reason. If machines other than the default are to have to have OWNER access, they probably should also have ADMINISTRATOR access. By granting machines ADMINISTRATOR access, they will automatically have OWNER access, given how OWNER access is set within the configuration.
The default access configuration is
HOSTALLOW_ADMINISTRATOR = $(CONDOR_HOST) HOSTALLOW_OWNER = $(FULL_HOSTNAME), $(HOSTALLOW_ADMINISTRATOR) HOSTALLOW_READ = * HOSTALLOW_WRITE = * HOSTALLOW_NEGOTIATOR = $(COLLECTOR_HOST) HOSTALLOW_NEGOTIATOR_SCHEDD = $(COLLECTOR_HOST), $(FLOCK_NEGOTIATOR_HOSTS) HOSTALLOW_WRITE_COLLECTOR = $(HOSTALLOW_WRITE), $(FLOCK_FROM) HOSTALLOW_WRITE_STARTD = $(HOSTALLOW_WRITE), $(FLOCK_FROM) HOSTALLOW_READ_COLLECTOR = $(HOSTALLOW_READ), $(FLOCK_FROM) HOSTALLOW_READ_STARTD = $(HOSTALLOW_READ), $(FLOCK_FROM)
This example configuration presumes that the condor_ collector and condor_ negotiator daemons are running on the same machine.
For each access level, an ALLOW or a DENY may be added.
Multiple machine entries in the configuration files may be separated by either a space or a comma. The machines may be listed by
To resolve an entry that falls into both allow and deny: individual machines have a higher order of precedence than wild card entries, and host names with a wild card have a higher order of precedence than IP subnets. Otherwise, DENY has a higher order of precedence than ALLOW. (this is how most people would intuitively expect it to work).
In addition, the above access levels may be specified on a per-daemon basis, instead of machine-wide for all daemons. Do this with the subsystem string (described in section 3.3.1 on Subsystem Names), which is one of: STARTD, SCHEDD, MASTER, NEGOTIATOR, or COLLECTOR. For example, to grant different read access for the condor_ schedd:
HOSTALLOW_READ_SCHEDD = <list of machines>
The following is a list of registered commands that daemons will accept. The list is ordered by daemon. For each daemon, the commands are grouped by the access level required for a daemon to accept the command from a given machine.
The command sent as a result of condor_ reconfig to reconfigure a daemon.
The command sent as a result of reconfig -full to perform a full reconfiguration on a daemon.
All commands that relate to a condor_ schedd daemon claiming a machine, starting jobs there, or stopping those jobs.
The command that condor_ checkpoint sends to periodically checkpoint all running jobs.
The command that condor_ preen sends to request the current state of the condor_ startd daemon.
The commands that a condor_ startd sends to the condor_ schedd when it must vacate its jobs and release the condor_ schedd's claim.
The commands which write information into the job queue (such as condor_ submit and condor_ hold). Note that for most commands which attempt to write to the job queue, Condor will perform an additional user-level authentication step. This additional user-level authentication prevents, for example, an ordinary user from removing a different user's jobs.
MASTER: All commands are registered with ADMINISTRATOR access:
This section provides examples of configuration settings. Notice that ADMINISTRATOR access is only granted through a HOSTALLOW setting to explicitly grant access to a small number of machines. We recommend this.
HOSTALLOW_ADMINISTRATOR = $(CONDOR_HOST) HOSTALLOW_OWNER = $(FULL_HOSTNAME), $(HOSTALLOW_ADMINISTRATOR)
HOSTALLOW_READ = *.ncsa.uiuc.edu HOSTALLOW_WRITE = *.ncsa.uiuc.edu HOSTALLOW_ADMINISTRATOR = $(CONDOR_HOST) HOSTALLOW_OWNER = $(FULL_HOSTNAME), $(HOSTALLOW_ADMINISTRATOR)
HOSTALLOW_WRITE = *.ncsa.uiuc.edu, *.math.uiuc.edu HOSTDENY_WRITE = lab-*.edu, *.lab.uiuc.edu, 177.55.* HOSTALLOW_ADMINISTRATOR = bigcheese.ncsa.uiuc.edu HOSTALLOW_OWNER = $(FULL_HOSTNAME), $(HOSTALLOW_ADMINISTRATOR)
\'' to continue a long list of machines onto multiple lines, making it more readable (this works for all configuration file entries, not just host access entries)
HOSTALLOW_READ = *.ncsa.uiuc.edu, *.cs.wisc.edu HOSTALLOW_WRITE = *.ncsa.uiuc.edu, raven.cs.wisc.edu HOSTALLOW_ADMINISTRATOR = $(CONDOR_HOST), bigcheese.ncsa.uiuc.edu, \ biggercheese.uiuc.edu HOSTALLOW_OWNER = $(FULL_HOSTNAME), $(HOSTALLOW_ADMINISTRATOR)
HOSTDENY_READ = *.mil HOSTALLOW_READ_SCHEDD = *.ncsa.uiuc.edu HOSTALLOW_WRITE = *.ncsa.uiuc.edu HOSTALLOW_ADMINISTRATOR = $(CONDOR_HOST), bigcheese.ncsa.uiuc.edu, \ biggercheese.uiuc.edu HOSTALLOW_ADMINISTRATOR_NEGOTIATOR = biggercheese.uiuc.edu HOSTALLOW_OWNER = $(FULL_HOSTNAME), $(HOSTALLOW_ADMINISTRATOR)
A new security feature introduced in Condor version 6.3.2 enables more fine-grained control over the configuration settings that can be modified remotely with the condor_ config_val command. The manual page for condor_ config_val on page details how to use condor_ config_val to modify configuration settings remotely. Since certain configuration attributes can have a large impact on the functioning of the Condor system and the security of the machines in a Condor pool, it is important to restrict the ability to change attributes remotely.
For each security access level described, the Condor administrator can define which configuration settings a host at that access level is allowed to change. Optionally, the administrator can define separate lists of settable attributes for each Condor daemon, or the administrator can define one list that is used by all daemons.
For each command that requests a change in configuration setting, Condor searches all the different possible security access levels to see which, if any, the request satisfies. (Some hosts can qualify for multiple access levels. For example, any host with ADMINISTRATOR permission probably has WRITE permission also). Within the qualified access level, Condor searches for the list of attributes that may be modified. If the request is covered by the list, the request will be granted. If not covered, the request will be refused.
The default configuration shipped with Condor is exceedingly restrictive. Condor users or administrators cannot set configuration values from remote hosts with condor_ config_val. Enabling this feature requires a change to the settings in the configuration file. Use this security feature carefully. Grant access only for attributes which you need to be able to modify in this manner, and grant access only at the most restrictive security level possible.
The most secure use of this feature allows Condor users to set attributes in the configuration file which are not used by Condor directly. These are custom attributes published by various Condor daemons with the <SUBSYS>_ATTRS setting described in section 3.3.5 on page . It is secure to grant access only to modify attributes that are used by Condor to publish information. Granting access to modify settings used to control the behavior of Condor is not secure. The goal is to ensure no one can use the power to change configuration attributes to compromise the security of your Condor pool.
The control lists are defined by configuration settings that contain SETTABLE_ATTRS in their name. The name of the control lists have the following form:
The two parts of this name that can vary are PERMISSION-LEVEL and the <SUBSYS>. The PERMISSION-LEVEL can be any of the security access levels described earlier in this section. Examples include WRITE, OWNER, and CONFIG.
The <SUBSYS> is an optional portion of the name. It can be used to define separate rules for which configuration attributes can be set for each kind of Condor daemon (for example, STARTD, SCHEDD, MASTER). There are many configuration settings that can be defined differently for each daemon that use this <SUBSYS> naming convention. See section 3.3.1 on page for a list. If there is no daemon-specific value for a given daemon, Condor will look for SETTABLE_ATTRS_PERMISSION-LEVEL .
Each control list is defined by a comma-separated list of attribute names which should be allowed to be modified. The lists can contain wild cards characters (`*').
Some examples of valid definitions of control lists with explanations:
SETTABLE_ATTRS_CONFIG = *Grant unlimited access to modify configuration attributes to any request that came from a machine in the CONFIG access level. This was the default behavior before Condor version 6.3.2.
SETTABLE_ATTRS_ADMINISTRATOR = *_DEBUG, MAX_*_LOGGrant access to change any configuration setting that ended with ``_DEBUG'' (for example, STARTD_DEBUG ) and any attribute that matched ``MAX_*_LOG'' (for example, MAX_SCHEDD_LOG ) to any host with ADMINISTRATOR access.
STARTD_SETTABLE_ATTRS_OWNER = HasDataSetAllows any request to modify the HasDataSet attribute that came from a host with OWNER access. By default, OWNER covers any request originating from the local host, plus any machines listed in the ADMINISTRATOR level. Therefore, any Condor job would qualify for OWNER access to the machine where it is running. So, this setting would allow any process running on a given host, including a Condor job, to modify the HasDataSet variable for that host. HasDataSet is not used by Condor, it is an invented attribute included in the STARTD_ATTRS setting in order for this example to make sense.
This topic is now addressed in more detail in section 3.7, which explains network communication in Condor.
On a Unix system, UIDs (User IDentification numbers) form part of an operating system's tools for maintaining access control. Each executing program has a UID, a unique identifier of a user executing the program. This is also called the real UID. A common situation has one user executing the program owned by another user. Many system commands work this way, with a user (corresponding to a person) executing a program belonging to (owned by) root. Since the program may require privileges that root has which the user does not have, a special bit in the program's protection specification (a setuid bit) allows the program to run with the UID of the program's owner, instead of the user that executes the program. This UID of the program's owner is called an effective UID.
Condor works most smoothly when its daemons run as root. The daemons then have the ability to switch their effective UIDs at will. When the daemons run as root, they normally leave their effective UID and GID (Group IDentification) to be those of user and group condor. This allows access to the log files without changing the ownership of the log files. It also allows access to these files when the user condor's home directory resides on an NFS server. root can not normally access NFS files.
If there is no condor user and group on the system, an administrator can specify which UID and GID the Condor daemons should use when they do not need root privileges in two ways: either with the CONDOR_IDS environment variable or the CONDOR_IDS configuration file setting. In either case, the value should be the UID integer, followed by a period, followed by the GID integer. For example, if a Condor administrator does not want to create a condor user, and instead wants their Condor daemons to run as the daemon user (a common non-root user for system daemons to execute as), the daemon user's UID was 2, and group daemon had a GID of 2, the corresponding setting in the Condor configuration file would be CONDOR_IDS = 2.2.
On a machine where a job is submitted, the condor_ schedd daemon changes its effective UID to root such that it has the capability to start up a condor_ shadow daemon for the job. Before a condor_ shadow daemon is created, the condor_ schedd daemon switches back to root, so that it can start up the condor_ shadow daemon with the (real) UID of the user who submitted the job. Since the condor_ shadow runs as the owner of the job, all remote system calls are performed under the owner's UID and GID. This ensures that as the job executes, it can access only files that its owner could access if the job were running locally, without Condor.
On the machine where the job executes, the job runs either as the submitting user or as user nobody, to help ensure that the job cannot access local resources or do harm. If the UID_DOMAIN matches, and the user exists as the same UID in password files on both the submitting machine and on the execute machine, the job will run as the submitting user. If the user does not exist in the execute machine's password file and SOFT_UID_DOMAIN is True, then the job will run under the submitting user's UID anyway (as defined in the submitting machine's password file). If SOFT_UID_DOMAIN is False, and UID_DOMAIN matches, and the user is not in the execute machine's password file, then the job execution attempt will be aborted.
While we strongly recommend starting up the Condor daemons as root, we understand that it is not always possible to do so. The main problems appear when one Condor installation is shared by many users on a single machine, or if machines are set up to only execute Condor jobs. With a submit-only installation for a single user, there is no need for (or benefit from) running as root.
What follows are the effects on the various parts of Condor of running both with and without root access.
In addition, some system information cannot be obtained without root access on some platforms (such as load average on IRIX). As a result, when running without root access, the condor_ startd must call other programs (for example, uptime) to get this information. This is much less efficient than getting the information directly from the kernel (which is what we do if we're running as root). On Linux and Solaris, we can get this information directly without root access, so this is not a concern on those platforms.
If you cannot have all of Condor running as root, at least consider whether you can install the condor_ startd as setuid root. That would solve both of these problems. If you cannot do that, you could also install it as a setgid sys or kmem program (depending on whatever group has read access to /dev/kmem on your system), and that would at least solve the system information problem.
Consider installing condor_ submit as a setgid condor program so that at least the stdout, stderr and UserLog files get created with the right permissions. If condor_ submit is a setgid program, it will automatically set it's umask to 002, and create group-writable files. This way, the simple case of a job that only writes to stdout and stderr will work. If users have programs that open their own files, they will need to know and set the proper permissions on the directories they submit from.
If you do choose to run Condor as non-root, then you may choose almost any user you like. A common choice is to use the condor user; this simplifies the setup because Condor will look for its configuration files in the condor user's directory. If you do not select the condor user, then you will need to ensure that the configuration is set properly so that Condor can find its configuration files.
If users will be submitting jobs as a user different than the user Condor is running as (perhaps you are running as the condor user and users are submitting as themselves), then users have to be careful to only have file permissions properly set up to be accessible by the user Condor is using. In practice, this means creating world-writable directories for output from Condor jobs. This creates a potential security risk, in that any user on the machine where the job is submitted can alter the data, remove it, or do other undesirable things. It is only acceptable in an environment where users can trust other users.
Normally, users without root access who wish to use Condor on their machines create a condor home directory somewhere within their own accounts and start up the daemons (to run with the UID of the user). As in the case where the daemons run as user condor, there is no ability to switch UIDs or GIDs. The daemons run as the UID and GID of the user who started them. On a machine where jobs are submitted, the condor_ shadow daemons all run as this same user. But if other users are using Condor on the machine in this environment, the condor_ shadow daemons for these other users' jobs execute with the UID of the user who started the daemons. This is a security risk, since the Condor job of the other user has access to all the files and directories of the user who started the daemons. Some installations have this level of trust, but others do not. Where this level of trust does not exist, it is best to set up a condor account and group, or to have each user start up their own Personal Condor submit installation.
When a machine is an execution site for a Condor job, the Condor job executes with the UID of the user who started the condor_ startd daemon. This is also potentially a security risk, which is why we do not recommend starting up the execution site daemons as a regular user. Use either root or a user (such as the user condor) that exists only to run Condor jobs.
Under Unix, Condor runs jobs either as the user that submitted the jobs, or as the user called nobody. Condor uses user nobody if the value of the UID_DOMAIN configuration variable of the submitting and executing machines are different.
When Condor cleans up after a executing a vanilla universe job, it does the best that it can by deleting all of the processes started by the job. Unfortunately, it is possible to fool Condor, and leave processes behind after Condor has cleaned up. If the job is running as user nobody, it is possible for it to leave a lurker process lying in wait for the next job run as nobody. The lurker process may prey maliciously on the next nobody user job, wreaking havoc.
Condor could prevent this problem by simply killing all processes run by the nobody user, but this would annoy many system administrators. The nobody user is often used for non-Condor system processes.
Condor provides a two-part solution to this difficulty. First, create user accounts specifically for Condor to use instead of user nobody. These can be low-privilege accounts, as the nobody user is. Create one of these accounts for each virtual machine per computer, so that distinct users can be used for concurrent processes. This prevents malicious behavior between processes running on distinct virtual machines. Section 3.12.7 details virtual machines. For a sample machine with two virtual machines, create two users that are intended only to be used by Condor. As an example, call them nobody1 and nobody2. Tell Condor about these users with the VMx_USER configuration variables, where x is replaced with the virtual machine number. In this example:
VM1_USER = nobody1 VM2_USER = nobody2
Reconfigure Condor, so that Condor will make use of these users instead of the nobody user. One more change is required to prevent lurker processes: tell Condor that these accounts are intended only to be used by Condor, so Condor can kill all the processes belonging to these users upon job completion. The configuration variable EXECUTE_LOGIN_IS_DEDICATED is introduced and set to True for this purpose.
EXECUTE_LOGIN_IS_DEDICATED = TRUE
Every executing process has a notion of its current working directory. This is the directory that acts as the base for all file system access. There are two current working directories for any Condor job: one where the job is submitted and a second where the job executes. When a user submits a job, the submit-side current working directory is the same as for the user when the condor_ submit command is issued. The initialdir submit command may change this, thereby allowing different jobs to have different working directories. This is useful when submitting large numbers of jobs. This submit-side current working directory remains unchanged for the entire life of a job. The submit-side current working directory is also the working directory of the condor_ shadow daemon. This is particularly relevant for standard universe jobs, since file system access for the job goes through the condor_ shadow daemon, and therefore all accesses behave as if they were executing without Condor.
There is also an execute-side current working directory. For standard universe jobs, it is set to the execute subdirectory of Condor's home directory. This directory is world-writable, since a Condor job usually runs as user nobody. Normally, standard universe jobs would never access this directory, since all I/O system calls are passed back to the condor_ shadow daemon on the submit machine. In the event, however, that a job crashes and creates a core dump file, the execute-side current working directory needs to be accessible by the job so that it can write the core file. The core file is moved back to the submit machine, and the condor_ shadow daemon is informed. The condor_ shadow daemon sends e-mail to the job owner announcing the core file, and provides a pointer to where the core file resides in the submit-side current working directory.