Peers,
This is my first post, thus must confess, I work professionally, between Dev Teams and Business Teams. Thus my coding skills and daily operations skills are limited, but my strength is around analyzing needs of end user and their workflows, followed by translating them into specific requirements or user stories (with technical constraints in mind) so that a dev team can work on them.
Problem Area:
Repetition of individual user workflow.
Each application have their own registration flow.
Imagine cell phone
Imagine screen recorder software
Imagine instead of recording screen - one could record user interaction.
Example:
Step 1 - âClickâ ( x , y ) [ icon.jpg ]
Step 2 - âClickâ ( x, y )
Step 3 - âtypeâ ( âabcâ )
Rationale: âAvoid repetition of user interaction by capturing user flow, to allow system to do it for youâ
Our phone interfaces are cluttered with apps and browsers with tabs. Many a times we find ourselves repeating steps on daily basis.
For example if User X has gone through the challenge of creating a new Account on a certain Web based App ( with X name, X last name, X phone etc ) then can the user Y - just click on a browser HTM plugin to do that for them!?
Honest Feedback and Criticism is appreciated.
p.s. - I am not married to my idea, so please plan on shooting it down or poking holes in it.
I may be missing your point but I use this app to automate my routine tasks:
I have been using it for years and it does know how to reach into inner-workings of windows controls rather than just push buttons if you need that level of control. It has a full programming language (it is BASIC but it gets the job done ) if you need some logic in your script.
Thank you @Bitking for drawing my attention to AutoIT, I will have to play with it. At high level seems like one has to deal with it own unique interface and language constructs.
How would one go about thinking of HTM encoder to capture Python Code (only containing this library) and then how could that encoder - decode for that?
Thought process - if one had such a python library paired with multi layer HTM encoder model, then a team could create sizable sample set, with manual intervention for training.
Followed by a meticulously designed app / browser plugin to catch user action translating into code. Which could perform that action!
It seems to me that you need a user interface that proposes to start tasks, maybe with parameters, as batches where the user only has to click âOKâ, or choose out of some options.
In that case I think that each task needs to become a set of bits in a custom defigned SDR. The HTM continuous learning would predict the next SDR state which can be translated in the next option in the user interface.
But donât you think you run the risk of upsetting users when their system continually predicts what they want to do? Wouldnât you feel like a drone if you had to do that all day?
