[DRMAA-WG] Conference call - Jul 28th - 19:00 UTC

[Peter Tröger]

Dear all,

the next DRMAA phone conference is scheduled on Jul 28th, at 19:00 UTC.

The phone conference line is sponsored by Oracle. Please consult the following page for dial-in numbers from your country:

http://www.intercall.com/oracle/access_numbers.htm

The conference code is 6513037. The security code is DRMAA (37622).

Preliminary meeting agenda:

1. Meeting secretary ?
2. Since job templates are now structs, should we do the same with ReservationTemplate ?
3. What shall we do with ReservationTemplate::nativeOptions ?
4. Add OS version to ReservationTemplate ?
5. What shall we do with email / blockEmail in JobTemplate ?
6. Sanity check for OS type description (check http://tinyurl.com/25cy9u7 )
7. Proposal for new JobInfo attributes: priority and memory consumption (check Google spreadsheet)
8. Is JobInfo::jobTemplate enough, or do we need to extract the supported job monitoring attributes from JobTemplate ? (check Google spreadsheet)
9. Is SGE really not able to distinguish between user mode CPU time and system mode CPU time ? (check Google spreadsheet)
10. Collaborative editing of Google spreadsheet

Google spreadsheet:
http://spreadsheets.google.com/ccc?key=0AqyvnBscJNqxcnJBSUs5dXRrU29EUVhGOGthc1lDTFE

Best regards,
Peter.


-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://www.ogf.org/pipermail/drmaa-wg/attachments/20100727/f99eeb20/attachment-0001.html

[Mariusz Mamoński]

Post by Peter Tröger
Dear all,
the next DRMAA phone conference is scheduled on Jul 28th, at 19:00 UTC.
The phone conference line is sponsored by Oracle. Please consult the
http://www.intercall.com/oracle/access_numbers.htm
The conference code is 6513037. ?The security code is DRMAA (37622).
1. Meeting secretary ?
2. Since job templates are now structs, should we do the same with ReservationTemplate ?
3. What shall we do with ReservationTemplate::nativeOptions ?
4. Add OS version to ReservationTemplate ?

I have just applied some changes to the DRMAA Advance Reservation
interface - i think it is now aligned with what we have now in the Job
reservation (mainly: ReservationTemplate as struct, nativeOptions as a
dictionary, getAllReservations() in monitoring session). So we can now
simply vote whether to accept those changes or not.

Post by Peter Tröger
5. What shall we do with email / blockEmail in JobTemplate ?
6. Sanity check for OS type description (check?http://tinyurl.com/25cy9u7 )
7. Proposal for new JobInfo attributes: priority and memory consumption
(check Google spreadsheet)
8. Is JobInfo::jobTemplate enough, or do we need to extract the supported
job monitoring attributes from JobTemplate ??(check Google spreadsheet)
9. Is SGE really not able to distinguish between user mode CPU time and
system mode CPU time ??(check Google spreadsheet)
10. Collaborative editing of Google spreadsheet
http://spreadsheets.google.com/ccc?key=0AqyvnBscJNqxcnJBSUs5dXRrU29EUVhGOGthc1lDTFE
Best regards,
Peter.
--
?drmaa-wg mailing list
?drmaa-wg at ogf.org
?http://www.ogf.org/mailman/listinfo/drmaa-wg

Cheers,

[Daniel Templeton]

One follow-up item from the meeting today. Since we're revisiting the
email attributes, can we talk about specifying when emails should be
sent? The OGE qsub command takes the -m switch, which can be composed
of the following:

`b' Mail is sent at the beginning of the job.
`e' Mail is sent at the end of the job.
`a' Mail is sent when the job is aborted or
rescheduled.
`s' Mail is sent when the job is suspended.
`n' No mail is sent.


Is that sort of functionality universal enough that we could expose it
through DRMAA?

Daniel

Post by Peter Tröger
Dear all,
the next DRMAA phone conference is scheduled on Jul 28th, at 19:00 UTC.
The phone conference line is sponsored by Oracle. Please consult the
http://www.intercall.com/oracle/access_numbers.htm
<http://www.intercall.com/oracle/access_numbers.htm>The conference code
is 6513037. The security code is DRMAA (37622).
1. Meeting secretary ?
2. Since job templates are now structs, should we do the same with ReservationTemplate ?
3. What shall we do with ReservationTemplate::nativeOptions ?
4. Add OS version to ReservationTemplate ?
5. What shall we do with email / blockEmail in JobTemplate ?
6. Sanity check for OS type description (check http://tinyurl.com/25cy9u7 )
7. Proposal for new JobInfo attributes: priority and memory consumption
(check Google spreadsheet)
8. Is JobInfo::jobTemplate enough, or do we need to extract the
supported job monitoring attributes from JobTemplate ? (check Google
spreadsheet)
9. Is SGE really not able to distinguish between user mode CPU time and
system mode CPU time ? (check Google spreadsheet)
10. Collaborative editing of Google spreadsheet
http://spreadsheets.google.com/ccc?key=0AqyvnBscJNqxcnJBSUs5dXRrU29EUVhGOGthc1lDTFE
Best regards,
Peter.
--
drmaa-wg mailing list
drmaa-wg at ogf.org
http://www.ogf.org/mailman/listinfo/drmaa-wg

[Mariusz Mamoński]

One follow-up item from the meeting today. ?Since we're revisiting the
email attributes, can we talk about specifying when emails should be
sent? ?The OGE qsub command takes the -m switch, which can be composed
? ? ? ? ? `b' ? ? Mail is sent at the beginning of the job.
? ? ? ? ? `e' ? ? Mail is sent at the end of the job.
? ? ? ? ? `a' ? ? Mail is sent when the job is aborted or
? ? ? ? ? ? ? ? ? rescheduled.
? ? ? ? ? `s' ? ? Mail is sent when the job is suspended.
? ? ? ? ? `n' ? ? No mail is sent.
Is that sort of functionality universal enough that we could expose it
through DRMAA?

Torque:
a - mail is sent when the job is aborted by the batch system.
b - mail is sent when the job begins execution.
e - mail is sent when the job terminates.

If the character "n" is specified, no normal mail is sent. Mail for
job cancels and other events outside of normal job processing are
still sent.

LSF:
mail can be sent on jobs start and/or termination.

so the common subset for the 3 systems seems to be notification on job
start/end.

Daniel

Post by Peter Tröger
Dear all,
the next DRMAA phone conference is scheduled on Jul 28th, at 19:00 UTC.
The phone conference line is sponsored by Oracle. Please consult the
http://www.intercall.com/oracle/access_numbers.htm
<http://www.intercall.com/oracle/access_numbers.htm>The conference code
is 6513037. The security code is DRMAA (37622).
1. Meeting secretary ?
2. Since job templates are now structs, should we do the same with ReservationTemplate ?
3. What shall we do with ReservationTemplate::nativeOptions ?
4. Add OS version to ReservationTemplate ?
5. What shall we do with email / blockEmail in JobTemplate ?
6. Sanity check for OS type description (check http://tinyurl.com/25cy9u7 )
7. Proposal for new JobInfo attributes: priority and memory consumption
(check Google spreadsheet)
8. Is JobInfo::jobTemplate enough, or do we need to extract the
supported job monitoring attributes from JobTemplate ? (check Google
spreadsheet)
9. Is SGE really not able to distinguish between user mode CPU time and
system mode CPU time ? (check Google spreadsheet)
10. Collaborative editing of Google spreadsheet
http://spreadsheets.google.com/ccc?key=0AqyvnBscJNqxcnJBSUs5dXRrU29EUVhGOGthc1lDTFE
Best regards,
Peter.
--
? ?drmaa-wg mailing list
? ?drmaa-wg at ogf.org
? ?http://www.ogf.org/mailman/listinfo/drmaa-wg

--
?drmaa-wg mailing list
?drmaa-wg at ogf.org
?http://www.ogf.org/mailman/listinfo/drmaa-wg

[Peter Tröger]

Condor:

notification=Always - eMail on job completion
notification=Complete - eMail on normal and abnormal job termination
notification=Error - eMail only on abnormal job termination

The documentation is flaky, and I could not manage to get an eMail on job start. You could implement this in the DRMAA library on the submission host by some log file analysis magic. Since the majority of relevant systems support job start information directly, I am tempted to agree to Mariusz proposal.

Best,
Peter.

Post by Mariusz Mamoński

Post by Daniel Templeton
One follow-up item from the meeting today. Since we're revisiting the
email attributes, can we talk about specifying when emails should be
sent? The OGE qsub command takes the -m switch, which can be composed
`b' Mail is sent at the beginning of the job.
`e' Mail is sent at the end of the job.
`a' Mail is sent when the job is aborted or
rescheduled.
`s' Mail is sent when the job is suspended.
`n' No mail is sent.
Is that sort of functionality universal enough that we could expose it
through DRMAA?

a - mail is sent when the job is aborted by the batch system.
b - mail is sent when the job begins execution.
e - mail is sent when the job terminates.
If the character "n" is specified, no normal mail is sent. Mail for
job cancels and other events outside of normal job processing are
still sent.
mail can be sent on jobs start and/or termination.
so the common subset for the 3 systems seems to be notification on job
start/end.

Post by Daniel Templeton
Daniel

Post by Peter Tröger
Dear all,
the next DRMAA phone conference is scheduled on Jul 28th, at 19:00 UTC.
The phone conference line is sponsored by Oracle. Please consult the
http://www.intercall.com/oracle/access_numbers.htm
<http://www.intercall.com/oracle/access_numbers.htm>The conference code
is 6513037. The security code is DRMAA (37622).
1. Meeting secretary ?
2. Since job templates are now structs, should we do the same with
ReservationTemplate ?
3. What shall we do with ReservationTemplate::nativeOptions ?
4. Add OS version to ReservationTemplate ?
5. What shall we do with email / blockEmail in JobTemplate ?
6. Sanity check for OS type description (check http://tinyurl.com/25cy9u7 )
7. Proposal for new JobInfo attributes: priority and memory consumption
(check Google spreadsheet)
8. Is JobInfo::jobTemplate enough, or do we need to extract the
supported job monitoring attributes from JobTemplate ? (check Google
spreadsheet)
9. Is SGE really not able to distinguish between user mode CPU time and
system mode CPU time ? (check Google spreadsheet)
10. Collaborative editing of Google spreadsheet
http://spreadsheets.google.com/ccc?key=0AqyvnBscJNqxcnJBSUs5dXRrU29EUVhGOGthc1lDTFE
Best regards,
Peter.
--
drmaa-wg mailing list
drmaa-wg at ogf.org
http://www.ogf.org/mailman/listinfo/drmaa-wg

--
drmaa-wg mailing list
drmaa-wg at ogf.org
http://www.ogf.org/mailman/listinfo/drmaa-wg

--
Mariusz
--
drmaa-wg mailing list
drmaa-wg at ogf.org
http://www.ogf.org/mailman/listinfo/drmaa-wg

[Peter Tröger]

Just to complete that: The DRMAAv2 state model has the according state groups "started" and "terminated":

http://wikis.sun.com/display/DRMAAv2/Data+Types

Therefore, my first proposal is the following modification:

struct JobTemplate {
...
attribute StringList email;
attribute boolean emailWhenStarted;
attribute boolean emailWhenTerminated;
...
}

Best,
Peter.

Post by Peter Tröger
notification=Always - eMail on job completion
notification=Complete - eMail on normal and abnormal job termination
notification=Error - eMail only on abnormal job termination
The documentation is flaky, and I could not manage to get an eMail on job start. You could implement this in the DRMAA library on the submission host by some log file analysis magic. Since the majority of relevant systems support job start information directly, I am tempted to agree to Mariusz proposal.
Best,
Peter.

Post by Mariusz Mamoński

Post by Daniel Templeton
One follow-up item from the meeting today. Since we're revisiting the
email attributes, can we talk about specifying when emails should be
sent? The OGE qsub command takes the -m switch, which can be composed
`b' Mail is sent at the beginning of the job.
`e' Mail is sent at the end of the job.
`a' Mail is sent when the job is aborted or
rescheduled.
`s' Mail is sent when the job is suspended.
`n' No mail is sent.
Is that sort of functionality universal enough that we could expose it
through DRMAA?

a - mail is sent when the job is aborted by the batch system.
b - mail is sent when the job begins execution.
e - mail is sent when the job terminates.
If the character "n" is specified, no normal mail is sent. Mail for
job cancels and other events outside of normal job processing are
still sent.
mail can be sent on jobs start and/or termination.
so the common subset for the 3 systems seems to be notification on job
start/end.

Post by Daniel Templeton
Daniel

Post by Peter Tröger
Dear all,
the next DRMAA phone conference is scheduled on Jul 28th, at 19:00 UTC.
The phone conference line is sponsored by Oracle. Please consult the
http://www.intercall.com/oracle/access_numbers.htm
<http://www.intercall.com/oracle/access_numbers.htm>The conference code
is 6513037. The security code is DRMAA (37622).
1. Meeting secretary ?
2. Since job templates are now structs, should we do the same with
ReservationTemplate ?
3. What shall we do with ReservationTemplate::nativeOptions ?
4. Add OS version to ReservationTemplate ?
5. What shall we do with email / blockEmail in JobTemplate ?
6. Sanity check for OS type description (check http://tinyurl.com/25cy9u7 )
7. Proposal for new JobInfo attributes: priority and memory consumption
(check Google spreadsheet)
8. Is JobInfo::jobTemplate enough, or do we need to extract the
supported job monitoring attributes from JobTemplate ? (check Google
spreadsheet)
9. Is SGE really not able to distinguish between user mode CPU time and
system mode CPU time ? (check Google spreadsheet)
10. Collaborative editing of Google spreadsheet
http://spreadsheets.google.com/ccc?key=0AqyvnBscJNqxcnJBSUs5dXRrU29EUVhGOGthc1lDTFE
Best regards,
Peter.
--
drmaa-wg mailing list
drmaa-wg at ogf.org
http://www.ogf.org/mailman/listinfo/drmaa-wg

--
drmaa-wg mailing list
drmaa-wg at ogf.org
http://www.ogf.org/mailman/listinfo/drmaa-wg

--
Mariusz
--
drmaa-wg mailing list
drmaa-wg at ogf.org
http://www.ogf.org/mailman/listinfo/drmaa-wg

--
drmaa-wg mailing list
drmaa-wg at ogf.org
http://www.ogf.org/mailman/listinfo/drmaa-wg

[Peter Tröger]

Participants: Mariusz, Roger, Dan, Peter
Special guest: Rayson Ho

The conference started late, since both co-chairs were late. Sorry to anybody who gave up before 19:15 UTC.

Post by Peter Tröger
1. Meeting secretary ?

Peter.

Post by Peter Tröger
2. Since job templates are now structs, should we do the same with ReservationTemplate ?

- Some recent adjustments of Mariusz in the wiki
- Reservation* structures are now aligned to latest developments in the Job* structures
- New MonitoringSession::getAllReservations() function, agreed by group, support in SGE approved
- Proposal by Mariusz to allow Reservation object fetching from Job object - agreed
- Proposal by Mariusz to introduce ReservationInfo as separate struct, similar to Job* structures - agreed

Post by Peter Tröger
3. What shall we do with ReservationTemplate::nativeOptions ?

- Already solved by Mariusz, ReservationTemplate now also supports drmsSpecific attribute

Post by Peter Tröger
4. Add OS version to ReservationTemplate ?

- No. Majority of systems just allow to request a general OS type, but not a specific variant

Post by Peter Tröger
5. What shall we do with email / blockEmail in JobTemplate ?

- blockEmail identified as (incorrectly generalized) adoption of old SGE feature - to be removed
- New semantic is that a missing eMail attribute in the job template leads to no eMail sending at all
- Status mails for the submitters user account (f.e. as in Condor) demands now explicit setting of the user name in the JT eMail attribute
- Roger: Explicit eMail address specification is demanded in the field (resolving issues on execution host)

Post by Peter Tröger
6. Sanity check for OS type description (check http://tinyurl.com/25cy9u7 )
7. Proposal for new JobInfo attributes: priority and memory consumption (check Google spreadsheet)
8. Is JobInfo::jobTemplate enough, or do we need to extract the supported job monitoring attributes from JobTemplate ? (check Google spreadsheet)
9. Is SGE really not able to distinguish between user mode CPU time and system mode CPU time ? (check Google spreadsheet)
10. Collaborative editing of Google spreadsheet
http://spreadsheets.google.com/ccc?key=0AqyvnBscJNqxcnJBSUs5dXRrU29EUVhGOGthc1lDTFE

Not covered due to time constraints.

Best regards,
Peter.

Post by Peter Tröger
Dear all,
the next DRMAA phone conference is scheduled on Jul 28th, at 19:00 UTC.
http://www.intercall.com/oracle/access_numbers.htm
The conference code is 6513037. The security code is DRMAA (37622).
1. Meeting secretary ?
2. Since job templates are now structs, should we do the same with ReservationTemplate ?
3. What shall we do with ReservationTemplate::nativeOptions ?
4. Add OS version to ReservationTemplate ?
5. What shall we do with email / blockEmail in JobTemplate ?
6. Sanity check for OS type description (check http://tinyurl.com/25cy9u7 )
7. Proposal for new JobInfo attributes: priority and memory consumption (check Google spreadsheet)
8. Is JobInfo::jobTemplate enough, or do we need to extract the supported job monitoring attributes from JobTemplate ? (check Google spreadsheet)
9. Is SGE really not able to distinguish between user mode CPU time and system mode CPU time ? (check Google spreadsheet)
10. Collaborative editing of Google spreadsheet
http://spreadsheets.google.com/ccc?key=0AqyvnBscJNqxcnJBSUs5dXRrU29EUVhGOGthc1lDTFE
Best regards,
Peter.

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://www.ogf.org/pipermail/drmaa-wg/attachments/20100729/7d9a0735/attachment.html