Scripts (Python): Getting up to speed with Python + sample script

Help & Support
1

Disclaimer
I have not thoroughly analysed everything Python-related in DOpus scripting - far from that. If you have some other problems/questions - ask, I will try to answer.

Intoduction
DOpus scripting capabilities are based on Microsoft COM and ActiveScripting technologies. DOpus support for them is geared towards languages that are "native" to ActiveScripting and bundled with Windows, ie. JScript and VBScript, much like some shell extensions are geared specifically towards Windows Explorer (making problems with DOpus). That is why the support causes some problems with "non-native" Python.

Components
I use python.org python and pywin32 on Windows 7 (all 64-bit versions) with 64-bit DOpus (free ActivePython is very weakly supported). No special setup is necessary.
Remember to always use .pys (PythonScript) file extension for DOPus Python scripts, otherwise scripting engine doesn't detect them.

Misc
When developing, keep PythonWin, Tools> Trace Collector debugging tool window open, to see extended problem information.
Also, 'import win32traceutil' in your script, to have more debug info displayed in the above window.

Issue 1 - Calling functions
Some functions exposed by DOpus cannot be properly called, or a call produces invalid result. My impression is that this is the case for functions supporting 0 or more arguments and functions in DOpus native datatype objects, like Vector.
You can observe this problem in two ways:

  • A script just silently dies on a line with function call. (In fact, it fails with com_error exception which is not propagated from Python; you can see it in Trace Collector debugging tool window.)
  • You get an unexpected exception in DOpus debug console (eg. AttributeError: .Blob) on a seemingly innocuous function call.
    Solution:
    Mark an object attribute as function before calling it:

object._FlagAsMethod('function') result = object.function()
or in real life:

DOpus._FlagAsMethod('Create') factory = DOpus.Create() factory._FlagAsMethod('Vector') myvec=factory.Vector()
instead of just

myvec=DOpus.Create().Vector()

Note, you need to flag each function just once per object lifetime. Flagging is a low cost (like putting value in a simple dictionary) and idempotent operation (you can execute it multiple times for a function with no side-effects).

Issue 2 - Indexing collections
Indexing collections with [] may not always work. You get TypeError: This object does not support enumeration in DOpus debug window. Just use () instead of []. This is actually normal when using pywin, and not specific to DOpus. So, you need to use:

for l in DOpus.listers: DOpus.Output(DOpus.listers(l).title)

Issue 3 - Default values
Some objects (eg. Toolbar) are described in documentation as having . To get this value, simply interpret the object using datatype of the value, eg.:

for t in DOpus.toolbars: DOpus.Output(DOpus.toolbars(t))
This gets toolbars' names: of a Toolbar object is a string.

Sample script

Below you can find a simple columns script, that uses MediaInfo library to fetch metadata from video files. MediaInfo is imo much faster than DOpus functions, and supports more video formats. But don't mix DOpus and MediaInfo video columns in one file panel, because it will make data display much slower!
This is not a polished piece of software (like I don't check whether MediaInfo.dll has actually loaded)- just something concrete to get started and play with.
Note, you will get English captions if your DOpus language is other than Polski.

4f8e64ae37a2b8021078de36a893c6698cc12b48.jpg832×51 12.1 KB

Script setup:
Download MediaInfo DLL from mediaarea.net/en/MediaInfo/Download/Windows - get DLL without installer. Put \MediaInfo.dll into %WINDIR%\system32, put \Developers\Source\MediaInfoDLL\MediaInfoDLL3.py into \Lib\site-packages\ (assumes Python 3)
Put the script as .pys file (removing .txt extension) in /dopusdata/Script AddIns
\Developers\List_Of_Parameters*.csv files contain information about possible values to play with.

And the script:
MediaInfo media columns.pys.txt (7.22 KB)
Have fun!

1 Like
2

[Merged into this thread. --Leo]

With Python as the scripting-language it seems not to be possible to filter the CommandList.

The following in jscript works as expected:

@Script jscript
function OnClick(click_data){

	comm = new Enumerator(DOpus.Create.Command.CommandList("u"));
	comm.moveFirst();

	while (comm.atEnd() == false) {

		DOpus.Output(comm.item());
		comm.moveNext();
	}
}

The same in Python does not work (but I got no exception):

[code]@Script Python
def OnClick(click_data):

comm = DOpus.Create.Command.CommandList('u')
for x in comm:
	DOpus.Output(x)[/code]

The same without providing the argument works fine:

[code]@Script Python
def OnClick(click_data):

comm = DOpus.Create.Command.CommandList
for x in comm:
	DOpus.Output(x)[/code]

There seems to be something wrong when calling the method with ()
In the following example, the 'Hello' does not appear in script output....

[code]@Script Python
def OnClick(click_data):

comm = DOpus.Create.Command.CommandList('u')
DOpus.Output('Hello')[/code]

The same:

[code]@Script Python
def OnClick(click_data):

comm = DOpus.Create.Command.CommandList('')
DOpus.Output('Hello')[/code]

The same:

[code]@Script Python
def OnClick(click_data):

comm = DOpus.Create.Command.CommandList()
DOpus.Output('Hello')[/code]

Any ideas?

3

Look up function calling issue here:

[Link to the root post was here. I have merged both threads so everything is in one place. --Leo]

4

Oh my god....

Thanks for the hint, Xyzzy.

The following works:

@Script Python
def OnClick(click_data):

	DOpus._FlagAsMethod('Create')
	factory = DOpus.Create()
	factory._FlagAsMethod('Command')
	my_command = factory.Command()
	my_command._FlagAsMethod('CommandList')
	my_commandlist = my_command.CommandList('u')
	
	for x in my_commandlist:
		DOpus.Output(x)

This is more then ugly :confused:

5

The best solution would be to have a fix on DO side (or probably providing a typelib), but Python is on the lower end of priorities queue due to low popularity among DO scripters (which is a catch-22 case, because it won't be gaining more if there are such problems).
The fastest solution would be imo to modify pywin to have DO-specific support, but this is a bad idea.
I have thought about creating some transparent proxy handling the issues, but this doesn't look to be simple.

6

Why is it up to us to fix a particular implementation of Python's mishandling of ActiveScripting conventions?

Have you asked them if they might fix it on their side?

Python is a low priority for us due to issues like this* indicating it is not well suited to ActiveScripting, plus the fact it will always require people install extra components to run scripts that could just as easily be written in Javascript or VBScript.

(* And several others we've spent time on over the months, despite predicting the issues would keep coming due to the language mismatch.)

If you use Python for ActiveScripting, it's never going to be ideal for you or for anyone who uses your scripts. Been saying this for months now. You're free to still choose to use it, but don't expect it to be perfect.

For smoother sailing, adapting to use another scripting language once you know one already is really not hard.

7

How nice it would be, to have a one-for-all-scripting-language that fits all daily demands.
I hoped that Python is such a language :frowning:

Because I don't want to have problems over problems with Python-Support in DOpus every day, I'll switch to JScript now :frowning:

On the other hand, JScript does not seem to be the perfect solution, too...

[quote]
DOpus Help 'Vector':
Some languages have better support than others for arrays, but the languages aren't consistent and some (like JScript) have incompatible arrays that Opus is unable to access at all[/quote].
:confused:

8

You don't want- don't change it. Having spent some time on the subject, I must say pywin approach is reasonable, and even provides workaround specifically for the calling function issue, which is the only significant nuisance.

[quote="leo"]
Have you asked them if they might fix it on their side?[/quote]

Things are OK on their side as far as I can tell. The best indication is probably that some DO functions work, and some don't. And the mechanism for detecting functions in pywin is the same for all of them. And I cannot find any logic in how DO COM works for the failing functions (which is only accidentally OK for VBS and JS - just because how they work internally).

When things go down to COM, no language is ideal (as Dinkelhopper's note indicates), because COM gives some abstraction only.

[quote="leo"]
For smoother sailing, adapting to use another scripting language once you know one already is really not hard.[/quote]

You are missing the whole point of re-using existing code, skills and capabilities. I think I will come out with some kind of transparent proxy resolving any major problems faster than getting to VBS or JS.

BTW, I think this horse have been beaten to death already, let's keep this topic technical.

9

Sorry, but this is just wrong. Python is incorrectly assuming methods are properties. You call a function like object.blah() and Python for some reason decides that blah is a property even when the presence of the brackets makes it obvious that it should be treated a method.

VBScript and JScript do not work "accidentally", they work because they have been written by people with at least some intelligence :slight_smile:

If you don't want to report this problem to pywin that's up to you, but you need to stop going on about it endlessly here since there's nothing we can do about it.

10

This is not how things work. Python checks, whether the blah really is a function, by querying COM for object properties, which you return wrong in some cases (with DISPATCH_PROPERTYGET - your objects should present themselves correctly - the FlagAsMethod was coded specifically for cases as yours, when object does not introduce itself correctly). BTW, if you used () for non-callable, you could end up with segfault when running unknown, external (COM) code. VBS and JS work, because [quote]Some languages cannot distinguish between retrieving a property and calling a method.[/quote] (https://msdn.microsoft.com/en-us/library/windows/desktop/ms221486(v=vs.85).aspx).

If you do it, do it right.

11

DISPATCH_PROPERTYGET is a flag you can set when calling an IDispatch interface. In this context it isn't something Opus sets but something that the scripting language sets when it calls into Opus, to say it's happy for a property to be called.

A caller can set both DISPATCH_PROPERTYGET and DISPATCH_METHOD at once if it doesn't care which is triggered and it is happy for either, leaving the choice up to the callee (Opus). Or the caller can explicitly request one or the other, presumably based on some knowledge it has of the object it is calling (which I do not thing applies in this case; see below) or based on having specific language syntax for calling each thing (like whether there are brackets after the name, as in C#), or because it wants to favor calling one and, if that fails, then fall back on the other.

Jon is the expert on this part of the code but as far as I know Opus does not indicate one way or another the type each named member is (except by failing if it is requested to only invoke the wrong type), on the assumption that the language calling it either won't care or will have the same behaviour and assumptions as VBScript and JScript. We don't ship an external type library or implement IDispatch::GetTypeInfo, which I think would be the ways that gets done. (Mainly because doing either would make adding small but useful things to the scripting interface, like the mouse/monitor stuff we just added, about twice as much work and thus a lot less likely to be done, and because it isn't needed for the main scripting languages and, at least we assumed, shouldn't really be needed by others.)

12

Here's an example of how Python is getting it wrong.

Remember, we don't have a type library. The only interface we have is pure IDispatch, which has two relevant functions:

GetIDsOfNames - takes a plain text "name" and returns a dispatch ID (or failure if the name is not known by the object it's called on).
Invoke - takes a dispatch ID (returned by GetIDsOfNames) and optional parameters, and invokes the method or property.

There is no provision in either of those functions for Opus to tell the caller that a particular name/ID is a "property" or "method". That information is passed from the caller to Opus, and is (or should be) determined by the context in which the name is used. As I've said before, there's really no difference (at least on our side) between a property and a method with 0 arguments.

This command was used as an example earlier so we'll use it again:

comm = DOpus.Create.Command.CommandList('u')

Now, here are the calls Opus sees to its IDispatch interface as a result of running this command in JScript:

000000CFC206D0C0: GetIDsOfNames ->Create<- returned 32788 000000CFC206D0C0: Invoke ->32788<- 000000CFCA2D6F20: GetIDsOfNames ->Command<- returned 0 000000CFCA2D6F20: Invoke ->0<- 000000CFCA383E60: GetIDsOfNames ->CommandList<- returned 32892 000000CFCA383E60: Invoke ->32892<- arg0 "u"

The first value in each line is the address of the object the methods are being called on. The first (000000CFC206D0C0) is the global "DOpus" object, and the subsequent addresses refer to new objects returned by each method invocation.

You can see that JScript first calls "Create", then "Command", and then finally "CommandList", passing it a single argument ("u").

Here's the same function when called from Python:

000000CFC206D8A0: GetIDsOfNames ->Create<- returned 32788 000000CFC206D8A0: Invoke ->32788<- 000000CFC1F480C0: GetIDsOfNames ->Command<- returned 0 000000CFC1F480C0: Invoke ->0<- 000000CFCA384EA0: GetIDsOfNames ->CommandList<- returned 32892 000000CFCA384EA0: Invoke ->32892<- 000000CFC20FC060: Invoke ->0<- arg0 "u"

Hopefully you can see the difference. Python calls "Create", then "Command", and then "CommandList" but it does not pass "u" as an argument to "CommandList". Instead, it takes the object returned by "CommandList" and then calls its default dispatch ID (0), passing the "u" to that instead.

Python is wrongly assuming that "CommandList" is a property (which doesn't have arguments) rather than a method (which does), and it ends up invoking the wrong method on the wrong object because of it.

There's nothing we can do to "fix" this on our side - it's 100% a bug in Python.

13

For the sake of completeness, here's the voice from the other side of the fence:

[quote]This is really an ugly situation.

They are wrong to say that this is "100% a bug in Python". It's partly
their fault as well. The problem is that the IDispatch mechanism is
ambiguous, and they have designed their object model in a way that
absolutely invites misinterpretation. Their CommandList object supports
calls with no parameters, it supports calls with one parameter, and it
supports a default dispatch. By being overly accommodating, technically
speaking, their object has publicly advertised that it supports Python's
interpretation (fetching a property result and calling its default
dispatch), and there is no way for them to say they prefer one over the
other. Python happens to prefer the default dispatch solution when
given a choice, and this is the result.[/quote]

14

IDispatch definitely leads to ambiguities. When they exist and there are two or more possible ways to do something, it seems best to choose the same one which the standard built-in languages (JScript / VBScript) use.

Of course, this is easier said than done as Microsoft do not document things in enough detail and edge cases like this are only discovered the hard way.

But it would also seem best to choose the path which does not make common/important things (like a method with optional arguments which returns an object) impossible, especially if it is to make something uncommon or mutually exclusive possible. (At least, I think it does. The details are quite subtle so I may be missing something.)

On our side, I am not sure it would be possible to make an method call with default arguments which returns an object and works with JScript/VBScript and with Python as well. We need that type of object, and we definitely need to remain compatible with JScript/VBScript, so I am not sure what we can do on our side.

CommandList's argument is entirely optional, and it returns a collection object. If the argument is given, it filters what is added to the collection object before that object is returned. If no argument is given then the returned collection is unfiltered and populated with the full set of items. On our side, we see a call to CommandList with zero arguments, and we treat that as a call to CommandList() and return the unfiltered collection object.

I don't think we have any way to know that the call to CommandList with zero arguments is really asking us for an object which can then be used to call CommandList(...) with arguments. That doesn't really make sense to me, and it is not what JScript/VBScript do. We have to pick one convention or the other, and we obviously must pick the one which JScript/VBScript use. If Python's aim is to slot-in as an alternative to JScript/VBScript, I think it would makes sense for it to work similarly.

There may be some confusion there. If you call CommandList() it returns an Opus StringSet collection object. (Similarly, if you call CommandList("U") you also get an Opus StringSet collection object.) It is the StringSet collection which supports default dispatch. This is the problem: Python is talking to completely the wrong object, passing a StringSet object arguments that are intended for an entirely different object's CommandList method. To pass arguments to the CommandList method, it should be calling the CommandList method with arguments.

If Python is to stay as it is then I can't see how this could work, since it would mean we cannot have any methods with optional arguments that return collections. (As before, the details are subtle so I am open to the possibility that I am wrong and have missed something.)