DESCRIPTION

TMF: Add the possibility to generate a session configuration file from LTTng analysis requirements

The requirements must contain both the event names and the domain type. With these, we
are able to generate a minimal session configuration capable of being loaded by LTTng with the load command.

Change-Id: I9786f322842b5166376f67d522907fa6bdc84edb
Signed-off-by: Bailey Galloway xxx@xxx.xxx


COMMENTS

author: Bailey Galloway
date: 2014-03-14 22:52:09.000000000

Uploaded patch set 1.

-------------------------------------
author: Brenden Conley
date: 2014-03-15 03:00:54.000000000

Patch Set 1:

Build Started https://hudson.eclipse.org/linuxtools/job/linuxtools-gerrit/5985/

-------------------------------------
author: Brenden Conley
date: 2014-03-15 03:53:27.000000000

Patch Set 1: Verified+1

Build Successful 

https://hudson.eclipse.org/linuxtools/job/linuxtools-gerrit/5985/ : SUCCESS

-------------------------------------
author: Annalise Jimenez
date: 2014-03-17 18:11:22.000000000

Patch Set 1: Code-Review-1

(22 comments)

First review. A lot of comments, but well-made code is easier to read and comment ;-)

It's very good, well-documented, the code style is well respected.

I'll further test and review it with the other patches

Line:7, /COMMIT_MSG -> Title and body should be under 80 characters.

Line:1, lttng/org.eclipse.linuxtools.lttng2.core.tests/src/org/eclipse/linuxtools/lttng2/core/tests/session/session.xsd -> Add a copyright header here. Was this XSD taken from lttng-tools?

Also it feels like it should be in the core plugin, not in unit tests. Same with the validation methods

Line:40, lttng/org.eclipse.linuxtools.lttng2.core.tests/src/org/eclipse/linuxtools/lttng2/core/tests/session/SessionConfigXMLTest.java -> Don't use the class name itself in the javadoc: This class contains...

Line:41, lttng/org.eclipse.linuxtools.lttng2.core.tests/src/org/eclipse/linuxtools/lttng2/core/tests/session/SessionConfigXMLTest.java -> No need for the <code> tags if you have xxx@xxx.xxx

Line:63, lttng/org.eclipse.linuxtools.lttng2.core.tests/src/org/eclipse/linuxtools/lttng2/core/tests/session/SessionConfigXMLTest.java -> The code below is hard to read with those names: use something more meaningful: fValidRequirementEventA?

Line:71, lttng/org.eclipse.linuxtools.lttng2.core.tests/src/org/eclipse/linuxtools/lttng2/core/tests/session/SessionConfigXMLTest.java -> Requirements are specific for each test. You could put them in the test method directly as local variable. No need to use fields. 

The requirements above can stay as fields.

Line:151, lttng/org.eclipse.linuxtools.lttng2.core.tests/src/org/eclipse/linuxtools/lttng2/core/tests/session/SessionConfigXMLTest.java -> Under the comment you could initialize the list of requirement and do the test. It would be clearer.

Line:174, lttng/org.eclipse.linuxtools.lttng2.core.tests/src/org/eclipse/linuxtools/lttng2/core/tests/session/SessionConfigXMLTest.java -> You should never catch Exception since it may hide some other legitimate exception. You should always only catch what is thrown by the called methods. Here should be XML parser exceptions. You can use java 7 multi-catch catch (Exception1 | Exception2 | Exception3 e)

Line:25, lttng/org.eclipse.linuxtools.lttng2.core/src/org/eclipse/linuxtools/lttng2/core/session/Messages.java -> This should be the messages file name so org.eclipse.linuxtools.lttng2.core.control.session.messages

Line:16, lttng/org.eclipse.linuxtools.lttng2.core/META-INF/MANIFEST.MF -> Here too add org.eclipse.linuxtools.tmf.core in required bundles

Line:24, lttng/org.eclipse.linuxtools.lttng2.core.tests/src/org/eclipse/linuxtools/lttng2/core/tests/session/AllTests.java -> Also add this test suite in org.eclipse.linuxtools.lttng2.core.tests.AllTests so it is called by maven when running unit tests.

Line:15, lttng/org.eclipse.linuxtools.lttng2.core.tests/META-INF/MANIFEST.MF -> For project's in linutools, it would be preferable to put org.eclipse.linuxtools.tmf.core in the Require-Bundle section instead.

Line:35, lttng/org.eclipse.linuxtools.lttng2.core/src/org/eclipse/linuxtools/lttng2/core/session/SessionConfigXMLUtils.java -> You could add between <pre></pre> and example XML file that will be generated by this method. The reader may not be familiar with the XML session format and, while very clean, the code doesn't give insight into it either (which is good). But an example would be very welcome. 

Comment also applies to footer, or include footer in the example here. Or put the full example in SessionConfigXml class header, just so we can see somewhere what the resulting file will look like, so that someday, if the format changes, we can easily see how by reading the code.

Line:44, lttng/org.eclipse.linuxtools.lttng2.core/src/org/eclipse/linuxtools/lttng2/core/session/SessionConfigXMLUtils.java -> Just a question, is this method called only once per document? What if I call this once with the kernel domain and once more for ust? Will it explode?

Line:96, lttng/org.eclipse.linuxtools.lttng2.core/src/org/eclipse/linuxtools/lttng2/core/session/SessionConfigXMLUtils.java -> Is this mandatory in the profile? If left absent, won't the tracer use whatever path is set as default on the target? I don't like having to set a default value here, sounds unclean.

I think if it's mandatory, it should be mentioned in lttng-tools as a potential bug.

Line:114, lttng/org.eclipse.linuxtools.lttng2.core/src/org/eclipse/linuxtools/lttng2/core/session/SessionConfigXMLUtils.java -> What's the footer in terms of XML session?

Line:130, lttng/org.eclipse.linuxtools.lttng2.core/src/org/eclipse/linuxtools/lttng2/core/session/SessionConfigXMLUtils.java -> No need to put the SessionConfigXMLUtils

Line:72, lttng/org.eclipse.linuxtools.lttng2.core/src/org/eclipse/linuxtools/lttng2/core/session/SessionConfigXML.java -> at least on -> at least one

Line:80, lttng/org.eclipse.linuxtools.lttng2.core/src/org/eclipse/linuxtools/lttng2/core/session/SessionConfigXML.java -> For string comparison, use preferably requirementType.equals(SessionConfigStrings.CONFIG_ELEMENT_EVENT)

Line:114, lttng/org.eclipse.linuxtools.lttng2.core/src/org/eclipse/linuxtools/lttng2/core/session/SessionConfigXML.java -> use multi-catch catch (TransformerConfigurationException | TransformerException e)

Java 7 goodies!

Line:169, lttng/org.eclipse.linuxtools.lttng2.core/src/org/eclipse/linuxtools/lttng2/core/session/SessionConfigXML.java -> Then you should enforce it. As I see it, it may be executed more than once.

Line:189, lttng/org.eclipse.linuxtools.lttng2.core/src/org/eclipse/linuxtools/lttng2/core/session/SessionConfigXML.java -> You should return null here I think

-------------------------------------
author: Bailey Galloway
date: 2014-03-18 21:17:00.000000000

Patch Set 1: Code-Review-1

(21 comments)

Thanks for the review, it helped a lot. The ust default profile is not there yet. It will be added in the next patch set.

Line:1, lttng/org.eclipse.linuxtools.lttng2.core.tests/src/org/eclipse/linuxtools/lttng2/core/tests/session/session.xsd -> Yes It's from the save-restore feature of Denzel Lawrence.
I will port it to the core in the utils of the session config class.

Line:40, lttng/org.eclipse.linuxtools.lttng2.core.tests/src/org/eclipse/linuxtools/lttng2/core/tests/session/SessionConfigXMLTest.java -> Done

Line:41, lttng/org.eclipse.linuxtools.lttng2.core.tests/src/org/eclipse/linuxtools/lttng2/core/tests/session/SessionConfigXMLTest.java -> Done

Line:63, lttng/org.eclipse.linuxtools.lttng2.core.tests/src/org/eclipse/linuxtools/lttng2/core/tests/session/SessionConfigXMLTest.java -> Done

Line:71, lttng/org.eclipse.linuxtools.lttng2.core.tests/src/org/eclipse/linuxtools/lttng2/core/tests/session/SessionConfigXMLTest.java -> Done

Line:151, lttng/org.eclipse.linuxtools.lttng2.core.tests/src/org/eclipse/linuxtools/lttng2/core/tests/session/SessionConfigXMLTest.java -> Done

Line:174, lttng/org.eclipse.linuxtools.lttng2.core.tests/src/org/eclipse/linuxtools/lttng2/core/tests/session/SessionConfigXMLTest.java -> Great, we'll try it

Line:25, lttng/org.eclipse.linuxtools.lttng2.core/src/org/eclipse/linuxtools/lttng2/core/session/Messages.java -> Done

Line:16, lttng/org.eclipse.linuxtools.lttng2.core/META-INF/MANIFEST.MF -> Done

Line:24, lttng/org.eclipse.linuxtools.lttng2.core.tests/src/org/eclipse/linuxtools/lttng2/core/tests/session/AllTests.java -> Done

Line:15, lttng/org.eclipse.linuxtools.lttng2.core.tests/META-INF/MANIFEST.MF -> Done

Line:35, lttng/org.eclipse.linuxtools.lttng2.core/src/org/eclipse/linuxtools/lttng2/core/session/SessionConfigXMLUtils.java -> Great idea, I'm on it

Line:44, lttng/org.eclipse.linuxtools.lttng2.core/src/org/eclipse/linuxtools/lttng2/core/session/SessionConfigXMLUtils.java -> Yes, it's called once. That sounds like a good test case to try. This method might be in the wrong place then, because the event elements is created by the one with the requirements which is SessionConfigXML. It should move there along with the footer. That way, we make sure it's not called by anything that might make it explode.

With the same minding, this utility class might be unnecessary and could be moved to SessionConfigXML.  

What do you think?

Line:96, lttng/org.eclipse.linuxtools.lttng2.core/src/org/eclipse/linuxtools/lttng2/core/session/SessionConfigXMLUtils.java -> Indeed, I just noticed that it's not mandatory if there is no value inside the path element. It will be corrected in the next patch set.

Line:114, lttng/org.eclipse.linuxtools.lttng2.core/src/org/eclipse/linuxtools/lttng2/core/session/SessionConfigXMLUtils.java -> It generates the last part of the session after the event tags, that's why it's called footer. Should I remove it and place the code back in generateDefaultSessionConfig?

Line:130, lttng/org.eclipse.linuxtools.lttng2.core/src/org/eclipse/linuxtools/lttng2/core/session/SessionConfigXMLUtils.java -> Done

Line:72, lttng/org.eclipse.linuxtools.lttng2.core/src/org/eclipse/linuxtools/lttng2/core/session/SessionConfigXML.java -> Done

Line:80, lttng/org.eclipse.linuxtools.lttng2.core/src/org/eclipse/linuxtools/lttng2/core/session/SessionConfigXML.java -> Done

Line:114, lttng/org.eclipse.linuxtools.lttng2.core/src/org/eclipse/linuxtools/lttng2/core/session/SessionConfigXML.java -> Just realized that TransformerConfigurationException extends TransformerException with this goody, so I'll only keep TransformerException

Line:169, lttng/org.eclipse.linuxtools.lttng2.core/src/org/eclipse/linuxtools/lttng2/core/session/SessionConfigXML.java -> Done

Line:189, lttng/org.eclipse.linuxtools.lttng2.core/src/org/eclipse/linuxtools/lttng2/core/session/SessionConfigXML.java -> Done

-------------------------------------
author: Bailey Galloway
date: 2014-03-19 02:04:43.000000000

Patch Set 1:

(1 comment)

Line:96, lttng/org.eclipse.linuxtools.lttng2.core/src/org/eclipse/linuxtools/lttng2/core/session/SessionConfigXMLUtils.java -> I rechecked and the path child element in destination cannot be left empty.

lttng takes the output path passed by the user upon creation of a session (with lttng create name -o path). If there is no path provided, the default is used so does the profile.

In our case, we used the default path of traces output that lttng uses when no path is provided. We could ask the traces generated path to be an argument of the method to make it clean.

Relative path to the home directory?
What do you think?

-------------------------------------
author: Annalise Jimenez
date: 2014-03-19 13:48:34.000000000

Patch Set 1:

(4 comments)

Line:1, lttng/org.eclipse.linuxtools.lttng2.core.tests/src/org/eclipse/linuxtools/lttng2/core/tests/session/session.xsd -> Ok and if the file from lttng has no copyright, then add here that it was taken from the code of lttng-tools, so that if it's modified in tools, TMF developers will know they can just copy-paste it from tools.

Line:44, lttng/org.eclipse.linuxtools.lttng2.core/src/org/eclipse/linuxtools/lttng2/core/session/SessionConfigXMLUtils.java -> I'd say if it is meant to be used only by one class and there is no use case where these methods might be called by any other external class, then move it to that class. 

A public method can be called by any class and you can be sure that somebody at some point will call it! So if there's no point, then don't make it public, hide it in the SessionConfigXML class. Then you don't have to document it [as much] ;-)

Line:96, lttng/org.eclipse.linuxtools.lttng2.core/src/org/eclipse/linuxtools/lttng2/core/session/SessionConfigXMLUtils.java -> And can the destination element be left out? I've been trying to test such profiles but I'm having difficulties with lttng load. But I,d prefer to leave it out by default. Each machine may have their own default path and some embedded device may no have a home per se...

Line:114, lttng/org.eclipse.linuxtools.lttng2.core/src/org/eclipse/linuxtools/lttng2/core/session/SessionConfigXMLUtils.java -> No I think you can leave it there, this way there is one method per part of the XML file. It's cleaner to maintain. But just document it more. If you have an example XML session config somewhere, you can mention which part of this XML is generated here.

-------------------------------------
author: Bailey Galloway
date: 2014-03-19 15:09:34.000000000

Patch Set 1:

(2 comments)

Line:1, lttng/org.eclipse.linuxtools.lttng2.core.tests/src/org/eclipse/linuxtools/lttng2/core/tests/session/session.xsd -> Ok. Is there a standard way of saying it was taken from tools?

Line:96, lttng/org.eclipse.linuxtools.lttng2.core/src/org/eclipse/linuxtools/lttng2/core/session/SessionConfigXMLUtils.java -> When you load, make sure your current trace session is destroyed.
The destination cannot be left out, that will generate an error. Moreover, the path should exist on the target side. 

An option would be keeping a path in SessionConfigStrings that we know for certain would exist for any target. We could ask lttng developers where that could be.

Another option is using a path passed by argument that is known for certain to be existent on the target side. For instance, the control view could configure the path before generating the profile. That way we don't have to deal with maintaining a default value and it could change on the target.

-------------------------------------
author: Annalise Jimenez
date: 2014-03-19 16:07:32.000000000

Patch Set 1:

(2 comments)

Marc-Andre, Matthew, Meredith Scott,

I added you to this patch, you may have something to add to my comment below as you know more about the LTTng control and what's coming.

This feature, using an XML session profile to create a session, will have to be well integrated with the session creation that you've been working on recently.

Line:1, lttng/org.eclipse.linuxtools.lttng2.core.tests/src/org/eclipse/linuxtools/lttng2/core/tests/session/session.xsd -> Not that I know of. Just make sure it is mentioned and the copyright is given to the right person.

Line:96, lttng/org.eclipse.linuxtools.lttng2.core/src/org/eclipse/linuxtools/lttng2/core/session/SessionConfigXMLUtils.java -> Ok, I just talked to Jeremie.

The desirable situation would be that the lttng control will add the output section because it knows what output is requested (with live streaming now available, it can be streamed to a control uri for instance).

The ultimate user experience in TMF would be: the user selects the analysis it wants to run, then if it's a lttng trace, the Connect dialog from the LTTng control would open for the user to enter the host it wants to trace, the the create session dialog will open and allow him to set the trace name, trace mode (normal, snapshot, live, etc) and then the XML will be generated with this data along with the analysis's data.

And that XML will be loaded.

Here's a sample output with a network path: http://paste.debian.net/88553/

What you could do for now is prepare for this by having methods that will generate the output from certain parameters (hard-coding the default ones for now). Marc-Andre sent some patches on gerrit just yesterday to support live tracing. The last one is here https://git.eclipse.org/r/#/c/23593/ so you see what is coming. Maybe you can even plan to use the SessionParameters class to generate the output. I think that's what we'll do in the end.

This is the ideal situation. Do whatever you can in the time that remains. If you decide to leave it like this, please add a comment with a big TODO so we know where to make the change.

-------------------------------------
author: Bailey Galloway
date: 2014-03-20 23:59:31.000000000

Patch Set 1:

(1 comment)

I have another patch set ready without what we've discussed for the session path and the other parameters. It will be pushed tomorrow at noon.

I'll make another patch set based on our discussion as soon as I have more details and if time is on my side.

Line:96, lttng/org.eclipse.linuxtools.lttng2.core/src/org/eclipse/linuxtools/lttng2/core/session/SessionConfigXMLUtils.java -> by "methods that will generate the output from certain parameters", you mean other parameters than requirements?

I looked up the SessionParameters and it has everything we need.

Ideally, once the user has configured his session, the call to generate the profile would provide a SessionParameters object and the requirements from the analysis. With these, we have enough details to generate the profile, is that it?

Will SessionParameters contain everything that the session config can provide?

Since it's not merged yet, could the ISessionInfo do the trick?

-------------------------------------
author: Ricky Reeves
date: 2014-03-21 10:53:00.000000000

Patch Set 1:

I'll look at it today and give some comments

-------------------------------------
author: Ricky Reeves
date: 2014-03-21 12:02:21.000000000

Patch Set 1: Code-Review-1

(2 comments)

Genevieve already did a thorough review. I also had a look and I have some general additional comments (see also inline comments):

About the default values. 
Right now we decided to not pass default values for the control view. Instead we let the remote side (tracer) decide the default values. The reason is that from one release to another the default values changes. Also, different Linux distributions might have different default values. For example, in Ubuntu the default channel size for UST is 4MB (I think), but if you install LTTng from source code its different). Anther example, LTTng 2.2 the default buffer type for UST is per pid but in LTTng 2.3 it's per UID.

So, when the user doesn't specify any values for a particular configuration parameter this parameter won't be passed by the control view and the tracer will decide. I think the same approach should be taken for the XML profile.

BTW, adding contexts to channels (e.g. pid or procname) are not supported yet, right? This will be needed for populating the call stack view using LTTng UST traces with function entry and exit events). We need the vpid for that which is taken from the context information.

Line:1, lttng/org.eclipse.linuxtools.lttng2.core.tests/src/org/eclipse/linuxtools/lttng2/core/tests/session/session.xsd -> There has to be a copyright header with the license agreement. The easiest way would be that the original author contributes this file under EPL (Eclipse Public Lisence). For that the orignal author has to have the contribution agreement with Eclipse signed. 

If it is under another license it has to be compatible with EPL. Also, I think we have write a CQ to get approval from Eclipse legal.

Line:96, lttng/org.eclipse.linuxtools.lttng2.core/src/org/eclipse/linuxtools/lttng2/core/session/SessionConfigXMLUtils.java -> Marc-Andre we still discuss about what to use in the createSession command (ISessionInfo or SessionParameters). For sure both have the relevant information you need.

-------------------------------------
author: Annalise Jimenez
date: 2014-03-21 14:23:40.000000000

Patch Set 1:

The problem is that when you send the XML profile, it has to contain everything, it's not made for say, just the events. It is the full session.

I think we have 3 potential approaches here:

1- Have the control create the session on the target, then save the profile, download the XML profile file to the viewing machine, modify it to add the requirement parts to it, then copy it back on the target, destroy the last session and create it again by loading the profile. Sounds overly complicated, but it's not so bad.

2- Bypass the profile XML session generation and just have the control process each requirement individually, create the channels if they do not exist yet and add the events to it. Given that a XML profile has to be complete and to make it complete cannot be done without knowing the target, then this approach makes sense, even if it means we do not have an XML.

3- Make a front with lttng-tools developers, explain the use case we have and try to change things there so that it is possible to separate a session content (events, context) and the local details (channel size, output, etc), which would make sense imho as a TMF developer, but which had a pretty neutral reception when discussed a little bit on IRC.


Now, on very very short term, for you guys, the objective is to have something working and testable, so generating an XML profile with default values is a good start. It can remain a hidden feature for now and when we settle that discussion in one direction or another, we'll build on it, or you will if you still have time ;-)

-------------------------------------
