Once you have an idea that an AI-based solution is appropriate for your task, you need to take an informal idea and turn it into a rigorous problem definition that can be solved. Below we give a few guidelines and examples for how you can work through this process.
If you have any questions about formalizing your AI problems, please feel free to reach out to us at any time!
Defining your Inputs and Outputs
Usually, the first step in building a machine learning-based AI system is defining what inputs you expect, and what outputs you want the system to produce. This process can be challenging, as it requires some experience to go from a general, somewhat vague idea of what goal you want to achieve to a more concrete, well defined-prediction task that can be solved by machine learning techniques.
In the below table, we provide a few examples, of more intuitive descriptions common AI tasks with corresponding inputs and expected outputs.
| Intent | Task | Input | Output |
|---|---|---|---|
| I want to tell if an SNS post is saying good or bad things about a product. | Sentiment Classification | Text | Sentiment Label (positive/negative) |
| I want to tell if an X-rayed lung displays signs of cancer or not. | Lung X-ray Classification | X-ray Image | Tumor Label (present/absent) |
| I want to turn speech into text for easy search or captioning. | Speech Recognition | Speech | Transcript |
| I want to create captions for images to assist blind users. | Image Captioning | Image | Caption |
| I want to automatically generate answers for questions in a chat bot. | Question Answering | Question | Answer |
| I want to create a robot to stock store shelves. | Robotic Control | Sensor Outputs | Actuator Values |
If you are handling a task that is already well-known in the AI R&D community, you may be in luck! There could very well be public datasets that you could use to evaluate or build an initial prototype of your system.
If you’re handling a task that is entirely new, you will have to define your inputs and outputs from scratch. However, given the broad span of AI research, it is highly likely that there are tasks that are at least adjacent to the one you are trying to solve, so it is usually prudent to find these related tasks and familiarize yourself with how they define their inputs and outputs, what data resources are available, etc.
If you’d like to try to find a task that corresponds to your usage scenario, you can get in contact below, and we’ll try to identify the task that most closely matches the one that you’re interested in:
Alternatively, you can click over to our AI Task List, which covers a broad variety of tasks in list format.
Auxiliary Outputs
In addition to the actual outputs of the AI system, there are often other auxiliary outputs that from a system that could be useful. For example:
- Confidence: How confident is the system in its prediction? This can be useful to (a) explain to a human consumer how likely the answer is to be correct, (b) to pass this confidence to some other system that uses the predictions downstream, or (c) abstain from making a prediction if the system is not confident enough.
- Explanations: What factors contributed to the system’s prediction? This can be useful for applications where the system needs to be able to explain its reasoning. These explanations can take a number of forms, with the most common being a “saliency” map, which shows which parts of the input were most important to the system’s prediction.
One good resource to read more on this topic is the Interpretable Machine Learning Book.
Deciding what a “Good” Output Is
As discussed in the “Should I Use AI?” section, you should have an idea of how an AI system will achieve value for the users of whatever system you are developing. A “good” output should be one that delivers on this expected value.
In some cases, the answer may seem obvious: in the case of lung X-ray classification for example, a “good” output would be one that correctly identifies whether or not there is a tumor present in the image. However, if a doctor is only provided with the label “tumor present” or “tumor absent”, they may not be able to make a decision on how to proceed with the patient. In this case, it may be necessary to provide an explanation highlighting the salient parts of the image so the doctor can look more closely and make their own determination about whether a tumor is present or not.
If you already have a deployed system with users, then this is an ideal scenario, as you can incorporate the AI system in the users’ workflow and see how it affects their satisfaction or productivity. This is typically done by A/B testing, where different segments of users use different versions of the system, and the results are compared. For instance:
- You test doctors on their ability to diagnose lung cancer using the AI system, and compare it to their ability to diagnose lung cancer using the same system without the AI component.
- You ask translators to translate text using the output of an automatic translation system and compare their efficiency with translators translating without help of a system.
If you don’t immediately have a deployed system, you should try to think back to your original value proposition, and use intuition to decide what exactly the outputs should look like.
Once you have an idea of what sorts of outputs will be achieving your goals, you will want to further come up with a way, ideally automatic, to measure whether the outputs of your system are in fact of high quality. We cover this more in the AI system evaluation section.