[DRMAA-WG] meeting minutes March 3rd 2010

[Daniel Gruber]

Attendees: Dan, Roger, Mariusz, Daniel

Meeting secretary: Daniel

1. JobMonitoring and JobTemplate list:
-> Please go all through the lists in the google documents and
select the important ones

http://spreadsheets.google.com/ccc?key=0AqyvnBscJNqxcnJBSUs5dXRrU29EUVhGOGthc1lDTFE&hl=en


2. JobStates
-> Merging suspended state to one state and have optional sub-states
like for hold state
-> Mariusz will compare terminated state with SAGA

3. Roger is working on IDL to C mapping

4. Thread safety: Taking out the deleteJobTemplate() (implementation
will delete it automatically, something like garbage collection when
not referenced anymore)?

Daniel: Checks implementation thread safety issues side of DRMAAv1 on SGE

Other thread safety points on agenda are deferred.

5. Move JobInfo into Job template. Roger, Dan, Daniel agreed in not to
merge.
(One reason was: Job object is alive and JobInfo information is static)
Explicit statement for DRMS that don't have information from master.
Up to
the implementation which fields are reported.

6. OGF meeting is in two weeks hence no conference call

Cheers

Daniel

[Peter Tröger]

Hi,

Post by Daniel Gruber
2. JobStates
-> Merging suspended state to one state and have optional sub-states
like for hold state

Changes applied to Wiki document.

Post by Daniel Gruber
4. Thread safety: Taking out the deleteJobTemplate() (implementation
will delete it automatically, something like garbage collection when
not referenced anymore)?

We had that discussion 4 years ago, see for example here:

http://www.ogf.org/pipermail/drmaa-wg/2006-June/thread.html

Personally, I would love to throw away all create..() and delete..() functions for data structures. Until now we left them in, because finalizers are no good replacement if you need predictable cleanup code in the library.

Meanwhile, the question is if any DRMAA C implementation actually has such code. For Condor, the answer is no, "create_job_template" and "destroy_job_template" are only used for the bookkeeping:

http://condor-ext.cvs.sourceforge.net/viewvc/condor-ext/condor_src/drmaa/auxDrmaa.c?view=markup

What about SGE and GridWay ? Do you guys have any relevant logic in "create_job_template" and "destroy_job_template" ??

The second argument in the past was explicit resource management for the application itself. This does not hold for anything but C.

In sum: I can live with having these functions only as optional addition in the C language binding, similar to the attribute setter and getter functions. This would mean that they are no mandatory part of the IDL spec.

Post by Daniel Gruber
5. Move JobInfo into Job template. Roger, Dan, Daniel agreed in not to
merge.
(One reason was: Job object is alive and JobInfo information is static)

'Static' somehow implies that JobInfo is only available in some terminal job state. This was the case in DRMAAv1, but now, we wanted to support also monitoring of running jobs. In this case, JobInfo also becomes alive.

Post by Daniel Gruber
Explicit statement for DRMS that don't have information from master.
Up to
the implementation which fields are reported.

I don't really understand this part.

Thanks,
Peter.


-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://www.ogf.org/pipermail/drmaa-wg/attachments/20100315/e85828e8/attachment-0001.html
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 2208 bytes
Desc: not available
Url : http://www.ogf.org/pipermail/drmaa-wg/attachments/20100315/e85828e8/attachment-0001.bin

[Daniel Gruber]

Post by Peter Tröger
Hi,

Post by Daniel Gruber
2. JobStates
-> Merging suspended state to one state and have optional sub-states
like for hold state

Changes applied to Wiki document.

Post by Daniel Gruber
4. Thread safety: Taking out the deleteJobTemplate() (implementation
will delete it automatically, something like garbage collection when
not referenced anymore)?

http://www.ogf.org/pipermail/drmaa-wg/2006-June/thread.html
Personally, I would love to throw away all create..() and delete..()
functions for data structures. Until now we left them in, because
finalizers are no good replacement if you need predictable cleanup
code in the library.
Meanwhile, the question is if any DRMAA C implementation actually has
such code. For Condor, the answer is no, "create_job_template" and
http://condor-ext.cvs.sourceforge.net/viewvc/condor-ext/condor_src/drmaa/auxDrmaa.c?view=markup
What about SGE and GridWay ? Do you guys have any relevant logic
in "create_job_template" and "destroy_job_template" ??
The second argument in the past was explicit resource management for
the application itself. This does not hold for anything but C.
In sum: I can live with having these functions only as optional
addition in the C language binding, similar to the attribute setter
and getter functions. This would mean that they are no mandatory part
of the IDL spec.

Post by Daniel Gruber
5. Move JobInfo into Job template. Roger, Dan, Daniel agreed in not to
merge.
(One reason was: Job object is alive and JobInfo information is static)

'Static' somehow implies that JobInfo is only available in some
terminal job state. This was the case in DRMAAv1, but now, we wanted
to support also monitoring of running jobs. In this case, JobInfo also
becomes alive.

Post by Daniel Gruber
Explicit statement for DRMS that don't have information from master.
Up to
the implementation which fields are reported.

I don't really understand this part.

The proposal was just to add information (or an explicit statement) to
the standard, that it is up to the implementation which fields
are updated (and if the fields are updated at all). Because, if
I remember correctly, that polling for information from client
side to master and master to execution host can create much overhead
(costs) and therefore from some implementation it can be better
(when they have no imformation at master side) to report nothing.

Daniel

Post by Peter Tröger
Thanks,
Peter.
--
drmaa-wg mailing list
drmaa-wg at ogf.org
http://www.ogf.org/mailman/listinfo/drmaa-wg

[Peter Tröger]

Hi,

Post by Daniel Gruber

Post by Peter Tröger

Post by Daniel Gruber
Explicit statement for DRMS that don't have information from master.
Up to
the implementation which fields are reported.

I don't really understand this part.

The proposal was just to add information (or an explicit statement) to
the standard, that it is up to the implementation which fields
are updated (and if the fields are updated at all). Because, if
I remember correctly, that polling for information from client
side to master and master to execution host can create much overhead
(costs) and therefore from some implementation it can be better
(when they have no imformation at master side) to report nothing.

This is now part of the JobInfo data structure description:

http://wikis.sun.com/display/DRMAAv2/Data+Types

Best,
Peter.


-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 2208 bytes
Desc: not available
Url : http://www.ogf.org/pipermail/drmaa-wg/attachments/20100329/bbddcd45/attachment.bin

[Daniel Templeton]

Post by Peter Tröger
Hi,

Post by Daniel Gruber
2. JobStates
-> Merging suspended state to one state and have optional sub-states
like for hold state

Changes applied to Wiki document.

Post by Daniel Gruber
4. Thread safety: Taking out the deleteJobTemplate() (implementation
will delete it automatically, something like garbage collection when
not referenced anymore)?

http://www.ogf.org/pipermail/drmaa-wg/2006-June/thread.html
Personally, I would love to throw away all create..() and delete..()
functions for data structures. Until now we left them in, because
finalizers are no good replacement if you need predictable cleanup
code in the library.
Meanwhile, the question is if any DRMAA C implementation actually has
such code. For Condor, the answer is no, "create_job_template" and
http://condor-ext.cvs.sourceforge.net/viewvc/condor-ext/condor_src/drmaa/auxDrmaa.c?view=markup
What about SGE and GridWay ? Do you guys have any relevant logic
in "create_job_template" and "destroy_job_template" ??
The second argument in the past was explicit resource management for
the application itself. This does not hold for anything but C.
In sum: I can live with having these functions only as optional
addition in the C language binding, similar to the attribute setter
and getter functions. This would mean that they are no mandatory part
of the IDL spec.

The SGE implementation does instead have logic in the create and
delete. Basically the create mallocs a data structure, and the delete
frees it. I thought we had decided, though, that the job templates
would be scoped to the session, so to explicitly delete the job
templates, terminate the session. We also said that we have to allow an
implementation to free job templates without warning if needed. If the
client tries to use a freed template, it gets an error.

Post by Peter Tröger

Post by Daniel Gruber
5. Move JobInfo into Job template. Roger, Dan, Daniel agreed in not to
merge.
(One reason was: Job object is alive and JobInfo information is static)

'Static' somehow implies that JobInfo is only available in some
terminal job state. This was the case in DRMAAv1, but now, we wanted
to support also monitoring of running jobs. In this case, JobInfo also
becomes alive.

Static means that the data is packed into an object that then no longer
changes. Let me demonstrate with an example. Assume there are two
attributes. If the attributes were merged into the JobTemplate object,
then code that gets the value of the first attribute followed by getting
the value of the second attribute, they might not both represent the
same state. If the attributes remain in the JobInfo object, and the
JobInfo object is static, then the attributes all represent the same
state. To update them to the current state, the client has to get a new
JobInfo object. Make sense?

Post by Peter Tröger

Post by Daniel Gruber
Explicit statement for DRMS that don't have information from master.
Up to
the implementation which fields are reported.

I don't really understand this part.

The point was just that an implementation should be allowed to not
support all monitoring attributes.

Daniel

Post by Peter Tröger
Thanks,
Peter.
--
drmaa-wg mailing list
drmaa-wg at ogf.org
http://www.ogf.org/mailman/listinfo/drmaa-wg

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://www.ogf.org/pipermail/drmaa-wg/attachments/20100315/064c7f99/attachment.html