UIElement SDK

From Screen Scraper Studio

Contents 1 Introduction 2 What is a UIElement object? 3 What is a string ID? 4 Creating the UIElement object 5 UIElement Properties 5.1 hwnd 6 UIElement Methods 6.1 GetID 6.2 InitializeFromID 6.3 InitializeFromScreenPoint 6.4 IsValid 6.5 WriteText 6.6 Activate 6.7 Click 6.8 IsInForeground

Introduction

UIElement has been designed with the goal to programatically manipulate user interface objects. User interface objects can be in various shapes and can be built using various technologies. UIElement aims to be a unique and simple interface to the myriad of UI objects you see on your screen.

What is a UIElement object?

This interface is used to abstract any visual object that the user can interact with (editable control, button, static text etc.). It works with major technologies to create UI, such as Windows common controls, GDI, HTML, .Net Forms, Java, WPF, Flash, Flex.

What is a string ID?

These string identifiers are used to uniquely associate the elements on the screen with our UIElement representation of them. Right now, they are subject to change, that's why we strongly recommend not to write them by hand, but use the Select tool in the ScreenScraper Studio (either Window Selection or Region Selection) to generate them and if needed change only the value of some attributes. Please note that you can use wildcards to define attributes.

Creating the UIElement object

Visual C++

In order to use the COM object in your C++ application, add the following import statement to your C++ source file, like this:

#import “<path>\UIElement.dll”

Optionally, you may add also:

using namespace UIElementLib;

in order to reference the objects without specifying the namespace.

Then create the object in one of the following two ways:

UIElementPtr obj(__uuidof(UIElement));

Or

UIElementPtr obj = NULL;

HRESULT hr = obj.CreateInstance(__uuidof(UIElement));

In order to destroy the object, call:

obj.Release();

Remark: before creating the object, CoInitialize API function must be called.

UIElement Properties

hwnd

Declared in UIElement interface.

LONG hwnd;

Specifies the handle of the window that visually contains the UIElement object.

UIElement Methods

GetID

Declared in UIElement interface.

HRESULT GetID([in] VARIANT_BOOL refresh, [out,retval] BSTR* result);

Computes the string ID of the current UIElement object.

Parameters

refresh - if set to VARIANT_TRUE, the function will recompute the ID. If the function is called for the first time, the ID is calculated anyway. 

InitializeFromID

Declared in UIElement interface.

HRESULT InitializeFromID([in] BSTR bstrID);

Initializes the internal data of the current object according to the given string ID.

InitializeFromScreenPoint

Declared in UIElement interface.

HRESULT InitializeFromScreenPoint([in] LONG x, [in] LONG y);

Initializes the internal data of the current object according to the user interface object located at the specified screen location.

IsValid

Declared in UIElement interface.

HRESULT IsValid([out,retval] VARIANT_BOOL* pbIsValid)
Checks whether the internal data of the object designates a valid user interface object.

WriteText

Declared in UIElement interface.

HRESULT WriteText([in] LONG method, [in] BSTR text);

Puts text in the UI control identified by the current UIElement object.

Parameters:

method - specifies the way that the text is output into the control. The possible values are: 
  UIE_WTM_NATIVE   = 0 - meaning native method, which puts the text using the API that the control belongs to. 
  UIE_WTM_SENDKEYS = 1 - meaning send keys method, which uses keyboard events to send the desired keys. 
text - specifies the text that the user want to put into the control. 
  It contains normal characters and special keys, such as Alt, Ctrl and so on. 
  Example: AbC[d(ctrl)]a[up(ctrl)k(alt)]"; this sample sequence outputs "AbC", then holds down (d) the Ctrl key, 
  presses "a" and then releases Ctrl (u) and hits Alt (k = press and release). Notice the [ and ( characters, 
  they are special characters for invoking special keys that cannot be specified as characters. 

Activate

Declared in UIElement interface.

HRESULT Activate();

Forces the parent window of the current UI element into the foreground and gives focus to the UI element.

Click

Declared in UIElement interface.

HRESULT Click([in] LONG dx, [in] LONG dy, [in] LONG flags);

Clicks the current UI element using the specified mouse button.

Parameters

dx - specifies the horizontal displacement of the clicking position. 
dy - specifies the vertical displacement of the clicking position. 
flags - indicates how the mouse click will be simulated. Its significance is the following: 
  Bits 0-1 - specify whether the click is single or double. The bit mask for this bit is UIE_CF_CLICK_MASK = 3. 
    The corresponding flag values are: 
    UIE_CF_SINGLE = 0 - a single click is issued. 
    UIE_CF_DOUBLE = 1 - double click. 
    UIE_CF_HOVER  = 2 - only change the mouse position, without clicking. 
  Bits 2-3 - they specify which button will be clicked. The bit mask for these bits is UIE_CF_BUTTON_MASK = 12. 
    The corresponding flag values are: 
    UIE_CF_LEFT   = 0 - the left mouse button is clicked. 
    UIE_CF_RIGHT  = 4 - right button. 
    UIE_CF_MIDDLE = 8 - middle button. 
    UIE_CF_SCREEN_COORDS = 16 - if this flag is set, the function interprets the given coordinates as screen coordinates. 
      If this flag is cleared, the coordinates are relative to the upper-left corner of the UI element. 
    UIE_CF_MOVE_CURSOR = 32 - if this flag is set, the position of the mouse cursor is moved to the clicking position. 

IsInForeground

Declared in UIElement interface.

HRESULT IsInForeground([out,retval] VARIANT_BOOL* pbRet);

Call this function to find out if the top level parent window of the current UI element is in the foreground.