FR2861192A1 - Machine e.g. ultrasonograph, interaction controlling method, involves displaying virtual control panel comprising controls of machine accessible by user, on tactile screen that occupies part of displaying screen of machine - Google Patents

Abstract

The method involves displaying a virtual control panel comprising all controls of a machine accessible by a user, on a tactile screen. The tactile screen occupies a part of a displaying screen of the machine. Buttons retrieving a predetermined list of values displayed by cross over effect, are used. A virtual alphanumeric keyboard for inputting characters is inserted in a touchpad (2) of the tactile screen. An independent claim is also included for a control interface for a machine.

Description

ERGONOMIC CONTROL METHOD AND DEVICE

  The present invention relates to a method and to an interactive ergonomic control device for a machine, in particular an ultrasound machine. BACKGROUND OF THE INVENTION

  In general, conventional ultrasound scanners use specific mechanical keyboards including physical controls to adjust their many operating parameters. These commands include simple buttons, rotary knobs, sliders, and trackballs (inverted mice). Such keyboards require large development budgets since they often use expensive molding tools, and suffer from a lack of flexibility, because the designer must identify from the outset the number and shape of the keys he is going to use.

  Sometimes, some ultrasound machines include touch screens that are integrated into the keyboard of the machine and are generally reserved for function keys that are thus reconfigurable according to the context in which the machine is located (for example, all Buttons displayed on the screen can be assigned to different functions if the machine is in operating mode with a B image, or if it is in Pulsed Doppler, Color Doppler or still image mode. In the general case, these touch screens are used in addition to a mechanical main keyboard as described above.

  In some cases, simple buttons are added near the display screen, buttons which are assigned functions assigned by the operating software of the machine. In this configuration, the user does not interact with the screen but uses physical keys whose function is displayed on the screen. This solution therefore does not require a touch screen, and can thus be satisfied with a simple conventional display screen.

  The subject of the present invention is a method of controlling a machine, such as an ultrasound machine, this machine comprising a screen displaying images that it develops, parameters relating to the operation of this machine being controllable by the user and being displayed by the image and / or in the form of software objects displayed on the screen, this method being a method that allows to eliminate any mechanical control, which offers great flexibility and allows quick and easy reconfiguration of commands.

  The present invention also relates to a machine control interface, such as an ultrasound machine, this machine comprising a screen displaying images that it develops, parameters relating to the operation of this machine being user-controllable and displayed by the image and / or in the form of software objects displayed on the screen, this interface must be ergonomic, simple to perform, easily reconfigurable, whatever the model of machine it equips.

  The method according to the invention is an interactive ergonomic control method of a machine comprising a screen at least part of which consists of a touch screen, screen on which are displayed the image developed by the machine and the corresponding information , and it is characterized in that it also displays on the touch part a virtual control panel comprising all the commands accessible to the user.

  According to another characteristic of the invention, when the machine is an ultrasound machine, the TGC is adjusted by a simple swipe of the finger on the touch screen reproducing the user's gesture on the alignments of physical cursors of the traditional machines. .

  According to yet another characteristic of the invention, by direct interaction with the ultrasound image, adjustment is made to at least one of the following parameters: depth scale, values displayed in the image zone, focal sliders, multidimensional objects such as zoom frame, Doppler gate, CFM window.

  The interface according to the invention is a control interface for a machine whose screen displays images developed by the machine and associated information, and it is characterized in that it comprises a touch screen occupying at least a part of the display screen, this touch screen being associated with means displaying on the touch screen virtual objects acting on the controls of the machine accessible to the user.

  According to another characteristic of the invention, the virtual objects 35 comprise at least one of the following objects: buttons, keys, set of numerical values, cursors, deformable curve representing a parameter of operation of the machine, touchpad, alphanumeric keyboard , at least a part of the images developed by the machine and displayed on the screen, scale representing a machine operating parameter, multidimensional objects, contextual menu tabs. Preferably, said set of numerical values is a set of predefined values, one of which is selectable, and the set of numerical values is superimposed.

  The present invention will be better understood on reading the detailed description of an embodiment, taken by way of nonlimiting example and illustrated by the appended drawing, in which: FIG. 1 is a front view of a interface according to the invention, the ultrasound image of which is separated from the control panel, FIGS. 2 and 3 are enlarged front views of part of the virtual control panel of FIG. 1, respectively not activated and activated with overprinting. 4 is a front view of the ultrasound image display part of the interface of the invention, showing in particular the TGC curve which the user can model. FIG. 5 is a front view of the interface of FIG. 1 showing in particular a Doppler gate, FIG. 6 is an enlarged partial view of the control panel of FIG. 5, Figure 7 e FIG. 8 is a front view of the display area of an ultrasound system with an interface according to the invention and a front view of the interface of FIG. display of a zoom frame, and Figure 9 is a front view of the display area of an ultrasound system with an interface according to the invention showing an ultrasound image and a Doppler gate. For ease of representation, this figure, like Figures 4 and 8, has been reproduced in negative.

  The present invention is described below with reference to an ultrasound system, which requires many commands accessible to the user, some of which are complex (they correspond to two-dimensional parameters, for example), but it is understood that it is not limited to this single application, and can be implemented in other applications, even if they have fewer user-accessible commands, such as, for example, SONAR consoles and systems. imaging systems such as multi-modal medical imaging consoles (MRI, digital radiography, nuclear medicine, etc.).

  The characteristic of the user interface constituted by the virtual control panel is to use a single screen to present both the ultrasound image (including the display near the image of the corresponding information which are the main operating parameters), and also, on its touch part (which can generally occupy all, for reasons of ease of manufacture), a control panel comprising the virtual buttons and buttons that are reconfigurable according to the context of use .

  The advantage of this solution is to offer an ergonomic user interface that does not require any physical keyboard or other external device such as conventional pointing devices (mouse or trackballs ..).

  The method of the invention comprises two different approaches: according to the first, it is deliberately chosen to give no control role to the so-called image zone (where the ultrasound image is presented) and the manipulations of the user are therefore limited the part of the screen dedicated to the virtual control panel only. This solution makes it possible to limit soiling carried on the display portion of the image. In this approach, the image areas and commands are said to be separate.

  According to the second approach, features of the displayed image are used to perform commands corresponding to the settings of the machine parameters. In this case, we obtain a much more intuitive user interface since the user will point in the image area the object on which he wishes to intervene. In this approach, image areas and commands are said superimposed.

  The image zone is defined as the part of the screen, with a black background in the present example, comprising in its central part the ultrasound image itself, and around the image various annotations such as date and time. , the names of the hospital and the patient, the values of the main parameters relating to the operating mode of the machine, a scale of depth, the electronic focal length of the system, a scale of gray levels, the model of the probe connected to the system, the application selected by the user, the depth of exploration, etc.

  The control area is defined as the part of the screen, all gray in this case, including all the commands available to the user such as buttons, sliders or virtual touchpad.

  There are several types of commands: dual-state buttons, buttons giving access to preset parameter lists, sliders for continuously adjusting one-dimensional quantities (such as gain for example), a virtual touchpad for adjusting simultaneously. two-dimensional quantities (such as the position in x and y of the Doppler gate for example), tabs for reassigning all the control functions of a part of the control area, etc. According to one embodiment of the invention, the most usual commands are positioned in the lower part of the screen for reasons of ergonomics for the user. In the left zone of this lower part, we have the six most usual controls (and this choice is made for each of the machine's operating modes), and these most usual commands are arranged in an arc around a larger command that is the freeze command that freezes the display on the current image, as shown in Figure 2. Indeed, this freeze command is very often used and must be very easy to activate by definition, since the user will activate it when viewing the desired image, which is usually fleeting. Typically, on a screen of 35 or 38 cm (usually 14 inches or 15 inches) diagonally, the implantation of the six buttons in an arc corresponds quite naturally to the position in the space of the ends of the fingers of the hand, because you just have to turn your wrist slightly to access these six buttons.

  The overprinting list of values is described now with reference to FIG. 3. The most usual commands are used to select most often parameters that can take a certain number of predetermined values (for example the depth of the image, the emission focal length, the excitation frequency of the probe, the emission power level, the logarithmic compression ratio of the image, etc ...). Rather than using the + and - buttons that should be repeatedly pressed to explore the list of possible choices, it is preferable to display the command-line image in a list of selectable values that at the same time become as many buttons that the user can press to validate his choice.

  For example, when the user presses the depth button in the P panel as shown in Figure 2 (B mode), the list of pre-defined depth values accessible to the user appears in a small window F overprint on the screen, as shown in Figure 3.

  The user can both view the set of available values and the value currently taken into account by the machine, and by simply pressing the desired value displayed in the frame appearing superimposed, change the setting of the machine. A display time (preset in a machine settings menu) allows this display to be overlaid after a few seconds if the user has not made a selection. Similarly, the user can delete this display by clicking outside the list window (or by pressing the surface of the touch screen in a neutral zone).

  These superimposed lists are also used in the upper part of the control panel and will in this case be arranged vertically, while when they are located in the lower part of the panel it is preferred to arrange them horizontally. These choices make it possible, on the one hand, not to mask the image part used for the visualization, and on the other hand to avoid that the user has to press the touch screen in the image zone which must as much as possible remain the most possible, or in any case to minimize the manual operation of the touch screen in the area displaying ultrasound images.

  The setting of the TGC is now described. The TGC is a complex setting that allows the user to change the gain of the B-picture to several different depths through independent sliders. In the case of conventional ultrasound scanners, this adjustment is carried out by means of a series of sliders (controlling potentiometers) arranged horizontally one above the other. To perform this adjustment of a single gesture on typically six to eight independent sliders, the users have the habit of tightening all the sliders between the palms of both hands and deforming globally the curve drawn by the sliders in the space . A perfectly vertical straight line is synonymous with a constant gain setting throughout the depth of the image.

  Moreover, it is usual to graph near the image a curve indicating the disposition of the sliders for each depth: this is called the TGC curve.

  The present invention uses the TGC curve shown graphically in the image area (see Figure 4, curve 1, which is substantially vertical) to adjust the position of each of the sliders (represented by small circles 1A on the curve 1). The user must grab a virtual cursor (by pressing the screen where the cursor is) and move it sideways from left to right and vice versa to vary the gain at the current depth. He can either release it to validate his choice, or, without releasing the pressure exerted on the screen, slide his finger vertically to the previous or next cursor to take control of this adjacent cursor. The system is programmed so that the control is given to the next cursor even if the finger is not rigorously applied to the cursor but in its excursion zone on the graph, which allows the user to shape the shape of the cursor. TGC curve by dragging its finger up and down and drawing the general shape of the curve with a single gesture on the screen. This realization makes it possible to offer to the user a method of adjustment as intuitive as it was on the conventional machines that relative to the series of juxtaposed sliders.

  This method of tuning TGC is strictly speaking to be classified in the so-called superimposed zone approach, even if it is usable in the separated zone approach.

  Another feature of the invention lies in the use of a virtual touchpad associated with the ultrasound image. In the separate zone approach in particular, and to avoid having to point directly to the ultrasound image to select objects or position pointers, an area 2 of the touch screen (see Figure 5) is programmed to be used as a touchpad, such as those that equip laptops for example. While the entire touch screen is used to record absolute scores (pressure in one place triggers the computer's reading of the x and y coordinates of the selected point), the touchpad is used to record relative pointer. In the current practice, a preliminary selection is necessary to initialize for example a process of positioning a pointer on the image (by the pressure of a key beforehand) then the user slides his finger on the virtual touchpad by a series of movements so as to position the pointer at the desired location of the ultrasound image. The advantage of touch pads of this type is to be able to multiply or overdrive the speed of movement of the cursor on the image according to the speed of movement of the finger on the virtual touchpad.

  In this case, the virtual touchpad is used to select parameters that can take a very large number of different values, such as the position in x and y of the zoom frame, the pulsed Doppler gate, the window CFM. The touchpad is flanked by six function keys that are typically two-state keys (turned on or off, whose activation is marked by a color change, for example), only one of which is activated at a time and indicates the parameter controlled by the touchpad.

  For example, by pressing the gate key in B mode, the Doppler gate appears on image B and the relative movements applied to the touchpad affect the x and y position of the Doppler gate on image B. On the image of Figure 5, the 3 gate button is activated and the Doppler door 4 can be positioned using the touchpad without touching the image.

  The virtual touchpad of the invention can also serve as an information display means. For example, the adjustment function that is activated by this touchpad can be displayed on this keypad using a symbol that allows the user to immediately view the function thus activated (instead to have to browse the neighboring buttons to detect which is lit).

  The interface of the invention also makes it possible to adjust amplification gains. In the lower right portion of the control panel (see Figures 1 and 5) are the Mode Selection BSM Buttons (B, Pulse Doppler PW and Color CFM Doppler, and Mode M), as well as C1 and C2 sliders for the gain settings, as seen in the enlargement of Figure 6. These sliders have the advantage of both select by sliding the cursor the desired value of the gain, but also to display the selected value, and this visually very ergonomic. These sliders can be used in different ways on the touch screen: 1. The user can press (select) the cursor and move it while maintaining the pressure exerted (classic technique of drag and drop).

  2. It can also press the high point or the low point of the scale 5 (see figure s 1, 5 and 7) according to whether it wishes to increase or decrease the gain, and this, repeatedly to shift the cursor 6 (figured in front of the scale 5) in successive steps.

  3. It can also press the high or low point while holding down the pressure to trigger the auto-repeat function, which will shift the cursor in successive steps until the finger pressure is released.

  When the operating mode of the machine is changed (for example from mode B to PW mode), the functions assigned to the sliders are changed (in PW mode, the sliders are assigned to the Doppler gain and audio gain settings).

  The invention also makes it possible to embed a virtual keyboard. When entering alphanumeric characters is necessary (to enter the name of the patient, the hospital, an application file or to enter an annotation on the image), the system superimposes a virtual alphanumeric keyboard 7 (see Figure 7) on the screen. The user can successively press the keys of this virtual keyboard as it would on a physical keyboard. Of course, this virtual keyboard is embedded in a position that does not hide the ultrasound image, or at least its portion exploited by the user when using this virtual keyboard.

  To allow the user to navigate among several different context menus, the interface of the invention uses graphical tabs 8 (see FIGS. 1, 5 and 7), which by simply pressing on the touch screen, at the location where is displayed the desired tab, allow to go from one menu to another. According to the present example, the upper part of the control panel, just under the tabs, contains adjustment buttons 9 specific to the current operating mode, but at any moment the user can for example decide to display the menu corresponding to a other mode by pressing the corresponding mode tab.

  In the approach with image areas and overlapping commands, we try to simplify the user interface and make it as intuitive as possible by allowing itself to intervene on the image area either with the finger on the touch screen, or with a external pointing device such as a mouse or a trackball.

  The invention is based on the principle of using all the objects (software objects) present in the image area (either on the image itself or among the information presented near the image) to allow the user to interact with the settings of the machine. These settings include in particular: A- Adjusting the image depth: Several objects are usable to consider an interaction with the user: 1. the low limit of the image area, 2. the bottom end of the scale depth, 3. the displayed value of the image depth.

  For example, if using the displayed image, the user can select the lower limit of the image by finger pressure in this lower part of the image, and while maintaining the pressure exerted, move your finger up or down (depending on whether you want to reduce or increase the depth). Depending on the amplitude of the displacement, the system presents on the screen the successive limits possible by a dotted line which allows the user to perceive the selected values. When the user releases his pressure, the selected value is validated.

  The same operation can be performed on the depth scale 5 by selecting its bottom end according to the same principles, but another use is possible by selecting any intermediate point of the scale 5. When this scale is selected in one any point, moving up or down changes the presentation of the scale by reducing or increasing the scale factor, as if the selected point was directly moved along the scale, o while not presenting pre-defined scales with a graphic coding different from the current scale (color change or dotted, for example). When the pressure is released, the selected scale is validated. This method makes it easier to visualize the change of low depth to significant depths.

  The displayed value of the image depth can also be used directly as a button: when the user selects it, a list of selectable values is superimposed in the image area and the user can directly select the desired value. This solution may temporarily mask part of the ultrasound image, but for a short time during which the user's attention is focused on the current setting.

  This last solution can be used for all the parameters that are permanently displayed in the image zone in the vicinity of the ultrasound image (for example general gain, frequency and transmission power, logarithmic compression law, etc. ) B- Adjustment of the emission focal length: The emission focal length of the ultrasound beam is permanently represented in the image area by a slider (representing the focus of the beam) positioned along the depth scale, and which can be operated by the user to change the position. Just select it and move it up or down. During this move, the system constantly displays the nearest selectable value (by a different color cursor, without erasing the original cursor) until the user validates his choice by releasing the pressure on the screen touch. The old cursor is then displayed and the selected position is validated by displaying the cursor in the new position. Adjustment of multi-dimensional objects: In this category of objects, we consider the Doppler gate, the zoom frame, the CFM window, which are objects defined by several parameters (position in x and y, dimension on one or two axes).

  1 - CFM window: In the case of the Color Doppler window (CFM), which is a complex frame since the user can adjust its position in x and y, its size in x and y, and, in the case of use with a linear probe, its angular position relative to the vertical of the image.

  Rather than using the touchpad for parameters two by two (each time after validating the choice of sizes by the pressure of a key position then size), the interface of the invention allows the user to come directly point to the object and depending on where it clicks (selection action), the system will recognize the action desired by the user. By clicking on one of the four corners of the window, it can vary the dimensions of the window in x and y according to the displacement on the screen, before releasing the pressure. By selecting any point inside the frame, it changes the position of the frame in x and y. By selecting the lateral segments (in the case of a linear probe), it is the angular position that will be modified by the sliding of the finger on the screen.

  2 - Zoom frame: The same principles can be applied to the zoom frame 25 which can be modified in size and position on the image B. An example of a zoom frame is shown at 10 in FIG.

  3- Doppler gate: The Doppler gate is a segment that the user positions on the image B to select the zone in which the Doppler analysis is performed. It is materialized by a line indicating the axis of fire and by two lines which delimit along this axis the temporal segment in which the analysis is carried out. By selecting any point located in the area of the door, the user can move it in the image B (in x and y), that is to say to modify at the same time the axis of fire and the position of the door along this axis. By selecting any point on the firing axis outside the door and near the door, the user can change the size of the door (which is the distance between the two lines shown orthogonally to the door). the axis of fire). In the case of linear probes, by selecting a point located in the lower part of the firing axis, the user can modify the angular position. An example of a Doppler gate is shown at 4 in FIG. 5 and at 11 in FIG.

Claims (16)

  1. An interactive ergonomic control method of a machine comprising a screen, at least a part of which consists of a touch screen, on which screen is displayed the image developed by the machine and the corresponding information, characterized in that the the touch panel also has a virtual control panel with all the commands accessible to the user.   2. Method according to claim 1, characterized in that one uses buttons calling lists of predetermined values displayed superimposed (Figure 3).   3. Method according to claim 1 or 2, characterized in that one uses a portion of the touch screen virtual touchpad (2).   4. Method according to claim 3, characterized in that the touchpad is a zone with relative displacements.   5. Method according to one of the preceding claims, characterized in that one uses virtual cursors (1A, 6).   6. Method according to one of the preceding claims, characterized in that is embedded in the area of the touch part of the screen a virtual keyboard (7) for entering characters.   7. Method according to one of the preceding claims, characterized in that one equips the virtual control panel of tabs (8) to be able to freely select several virtual panels arranged at the same position on the screen.   8. Method according to one of the preceding claims, characterized in that to make adjustments of the parameters of the machine, it acts by direct interaction with the image.   9. Method according to one of the preceding claims, characterized in that the machine is an ultrasound machine.   10. A method according to claim 9, characterized in that an adjustment of the TGC is carried out by a simple swipe of the finger on the touch screen reproducing the gesture of the user on the alignments of physical cursors of the traditional machines. .   11. Method according to claim 10, characterized in that by direct interaction with the ultrasound image, the adjustment is made at least one of the following parameters: depth scale, values displayed in the image zone, focal sliders. , multidimensional objects such as zoom frame, Doppler gate, CFM window.   12. Control interface for a machine whose screen displays images developed by the machine and associated information, characterized in that it comprises a touch screen occupying at least a portion of the display screen, this touch screen being associated with means displaying on the touch screen virtual objects acting on the controls of the machine accessible to the user.   13. Interface according to claim 12, characterized in that the virtual objects comprise at least one of the following objects: buttons, keys, set of numerical values, sliders, deformable curve representing an operating parameter of the machine, touchpad, alphanumeric keypad, at least a part of the images developed by the machine and displayed on the screen, scale representing a machine operating parameter, multidimensional objects, contextual menu tabs.   Interface according to claim 13, characterized in that said set of numerical values is a set of predefined values one of which is selectable.   15. Interface according to claim 14, characterized in that the set of numerical values is displayed superimposed.   Interface according to one of claims 13 to 15, characterized in that the virtual touchpad also serves as an information display means.