I am switching gears to discuss my least favorite testing activity: Writing test cases! Personally, I would rather test the application than write the test cases. I have used automation for UI testing, web service testing, and data generation. With the rise of artificial intelligence tools, why not utilize them for testing activities to be more efficient? Whether you think about it or not, we already use AI tools to be more efficient. Spell-checking comes to mind, and Grammarly keeps underlining words and popping up windows as I type! I started using ChatGPT for test case generation to see how useful it could be.
Using ChatGPT for Test Case Generation
I first mocked up requirements for a few of the fields on the BlazeDemo Purchase page:
| Field Name | Required? | Min Length | Max Length | Database | Notes |
| Name | Y | 1 | 50 | Purchase.Name | |
| Address | Y | 1 | 100 | Purchase.Address | |
| City | Y | 1 | 50 | Purchase.City | |
| State | Y | 1 | 10 | Purchase.State | Populated list from State table. State.Abbreviation |
| Purchase | When the user clicks button, information is saved in the Purchase table. Validation messages are validated on both the client side and server side. Validation messages are “{Field Name} is required.” if value is blank. |
Here is the initial conversation I had with ChatGPT. Note: I just copied the data from the table above and pasted it in the prompt (misspellings and all) and asked “Can you write test cases for these?”.
ChatGPT Prompt:
I have these requirements: Field Name Required? Min Length Max Length Database Notes Name Y 1 50 Purchase.Name Address Y 1 100 Purchase.Address City Y 1 50 Purchase.City State Y 1 10 Purchase.State Populated list from State table. State.Abrreviation Purchase When the user clicks button, information is saved in the Purchase table. Validation messages are validated on both the client side and server side. Validation messages are “{Field Name} is required.” if value is blank. Can you write test cases for these?
Response:
Certainly! Below are some test cases that cover the requirements you've provided. These test cases are designed to validate both the client-side and server-side functionalities for the fields and the Purchase button.
Test Cases for Purchase Form
Test Case 1: Validate "Name" Field
1. Precondition: User is on the Purchase form.
2. Action: Leave the "Name" field blank.
3. Expected Result: Validation message "Name is required." appears.
Test Case 2: Validate "Name" Field Length
1. Precondition: User is on the Purchase form.
2. Action: Enter more than 50 characters in the "Name" field.
3. Expected Result: Validation message "Name exceeds maximum length of 50 characters." appears.
Test Case 3: Validate "Address" Field
1. Precondition: User is on the Purchase form.
2. Action: Leave the "Address" field blank.
3. Expected Result: Validation message "Address is required." appears.
Test Case 4: Validate "Address" Field Length
1. Precondition: User is on the Purchase form.
2. Action: Enter more than 100 characters in the "Address" field.
3. Expected Result: Validation message "Address exceeds maximum length of 100 characters." appears.
Test Case 5: Validate "City" Field
1. Precondition: User is on the Purchase form.
2. Action: Leave the "City" field blank.
3. Expected Result: Validation message "City is required." appears.
Test Case 6: Validate "City" Field Length
1. Precondition: User is on the Purchase form.
2. Action: Enter more than 50 characters in the "City" field.
3. Expected Result: Validation message "City exceeds maximum length of 50 characters." appears.
Test Case 7: Validate "State" Field
1. Precondition: User is on the Purchase form.
2. Action: Leave the "State" field blank or unselected.
3. Expected Result: Validation message "State is required." appears.
Test Case 8: Validate "State" Field Options
1. Precondition: User is on the Purchase form.
2. Action: Check the options available in the "State" dropdown.
3. Expected Result: All options should match the abbreviations in the State table.
Test Case 9: Validate Purchase Button with All Fields Empty
1. Precondition: User is on the Purchase form.
2. Action: Click the Purchase button without filling in any fields.
3. Expected Result: Validation messages for all required fields appear.
Test Case 10: Validate Purchase Button with All Fields Filled
1. Precondition: User is on the Purchase form.
2. Action: Fill in all fields correctly and click the Purchase button.
3. Expected Result: Information is saved in the Purchase table, and no validation message appears.
These test cases should provide a comprehensive validation of the Purchase form based on the requirements you've specified. Feel free to adapt them as needed.
The overall results were good I thought, and the format of the test cases was acceptable since I never mentioned a format.
I ran the prompt again and asked ChatGPT to put the test case responses in an XML structure (more on that later), and it provided five test cases this time. I mentioned one more specific test case, and it generated it. Six in all. I was pleased with this second set of test case results as well. (I am not listing them here because this post is going to be long.)
Azure’s Create Work Item API
I then looked into how it could create test cases from Microsoft’s API documentation for the Create Work Item API for Azure. You can find that here: Work Items – Create – REST API (Azure DevOps Work Item Tracking) | Microsoft Learn
ChatGPT Prompt:
I have this API that needs testing. Can you write the Test cases for it? Can you output the test cases in this XML structure? (and then I pasted in the top 1/3 of that page, from the top down to “Examples Create work item Sample Request”, including the sample request.
Response:
<TestCases>
<!-- Test Case 1: Create a Basic Work Item -->
<TestCase>
<Title>Create Basic Work Item</Title>
<Description>Test the creation of a basic work item with minimal required fields.</Description>
<Precondition>API is up and running. OAuth2 token is available.</Precondition>
<Steps>
<Step>
<Action>POST request to the API with minimal required fields (organization, project, type, api-version).</Action>
<ExpectedResult>200 OK response and a work item is created.</ExpectedResult>
</Step>
</Steps>
<SummaryDescription>Work item should be created successfully with minimal required fields.</SummaryDescription>
</TestCase>
<!-- Test Case 2: Create Work Item with All Optional Parameters -->
<TestCase>
<Title>Create Work Item with All Optional Parameters</Title>
<Description>Test the creation of a work item with all optional parameters.</Description>
<Precondition>API is up and running. OAuth2 token is available.</Precondition>
<Steps>
<Step>
<Action>POST request to the API with all optional parameters ($expand, bypassRules, suppressNotifications, validateOnly).</Action>
<ExpectedResult>200 OK response and a work item is created with the specified optional parameters.</ExpectedResult>
</Step>
</Steps>
<SummaryDescription>Work item should be created successfully with all optional parameters.</SummaryDescription>
</TestCase>
<!-- Test Case 3: Create Work Item with Invalid Organization -->
<TestCase>
<Title>Create Work Item with Invalid Organization</Title>
<Description>Test the creation of a work item with an invalid organization name.</Description>
<Precondition>API is up and running. OAuth2 token is available.</Precondition>
<Steps>
<Step>
<Action>POST request to the API with an invalid organization name.</Action>
<ExpectedResult>Non-200 response with an error message indicating the invalid organization.</ExpectedResult>
</Step>
</Steps>
<SummaryDescription>API should return an error for invalid organization name.</SummaryDescription>
</TestCase>
<!-- Test Case 4: Create Work Item with Missing Required Fields -->
<TestCase>
<Title>Create Work Item with Missing Required Fields</Title>
<Description>Test the creation of a work item with missing required fields.</Description>
<Precondition>API is up and running. OAuth2 token is available.</Precondition>
<Steps>
<Step>
<Action>POST request to the API without one or more required fields (organization, project, type, api-version).</Action>
<ExpectedResult>Non-200 response with an error message indicating the missing required fields.</ExpectedResult>
</Step>
</Steps>
<SummaryDescription>API should return an error for missing required fields.</SummaryDescription>
</TestCase>
<!-- Test Case 5: Create Work Item with Invalid OAuth2 Token -->
<TestCase>
<Title>Create Work Item with Invalid OAuth2 Token</Title>
<Description>Test the creation of a work item with an invalid OAuth2 token.</Description>
<Precondition>API is up and running.</Precondition>
<Steps>
<Step>
<Action>POST request to the API with an invalid OAuth2 token.</Action>
<ExpectedResult>Non-200 response with an error message indicating the invalid OAuth2 token.</ExpectedResult>
</Step>
</Steps>
<SummaryDescription>API should return an error for invalid OAuth2 token.</SummaryDescription>
</TestCase>
</TestCases>
I think these test cases are a good start. I followed up and asked, “Can you create a test case with specific data for creating an Epic?”
Response:
<TestCases>
<!-- Test Case: Create an Epic Work Item with Specific Data -->
<TestCase>
<Title>Create an Epic Work Item</Title>
<Description>This test case validates the API's ability to create an Epic work item with specific data.</Description>
<Precondition>Azure DevOps project and authentication token must be available. The project should support Epic work items.</Precondition>
<Steps>
<Step>
<Action>Prepare a POST request with the following details:
- organization: "YourOrganization"
- project: "YourProject"
- type: "Epic"
- API version: "7.1-preview.3"
- Request Body:
[
{
"op": "add",
"path": "/fields/System.Title",
"from": null,
"value": "Sample Epic"
},
{
"op": "add",
"path": "/fields/System.Description",
"from": null,
"value": "This is a sample Epic for testing."
}
]
</Action>
<ExpectedResult>Receive a 200 OK response with the created Epic work item's details, including the title "Sample Epic" and the description "This is a sample Epic for testing."</ExpectedResult>
</Step>
</Steps>
<SummaryDescription>An Epic work item should be created successfully with the specified title and description.</SummaryDescription>
</TestCase>
</TestCases>This test case is also good.
Using AI
We use artificial intelligence in our work already. Besides spelling and grammar checkers, there are Visual Studios’ IntelliSense, code generators, and customer service chatbots. Ok, that last one’s not too helpful some of the time! The image at the top of this post was generated from DALL·E 2.
Using ChatGPT for test case generation makes sense. However, you might be asking what’s the point if these test cases still need to be entered into a project management tool like Azure Boards or Jira. That will be the topic of the next post.