Azure AI Developer Hackathon

I’ve just completed participating in the Azure AI Developer Hackathon that was looking to provide participants an opportunity to gain hands-on experience building intelligent applications on the Azure platform, supercharged by the power of GitHub Copilot.

My submission is an Owner Builder AI Assistant.

Inspiration:

The inspiration for the Owner Builder AI Assistant stemmed directly from my own experience as a first-time owner builder. I’m currently embarking on the journey of a building project, and the reality of managing the project myself quickly became apparent. One of the most time-consuming and often frustrating aspects has been the process of requesting quotes from various trades. Ensuring these quotes are accurate, comprehensive, and aligned with building standards feels like a constant uphill battle, especially with limited prior experience. I envisioned a tool that could act as a digital assistant, guiding me through these complexities and empowering me to manage my construction project with greater confidence and efficiency.

The Owner Builder AI Assistant assists by generating prompts for AI based on your project phase and tasks that are then used in conjunction with building standards to generate either how to do the job yourself or what to ask for from a contractor. It then allows you to upload your approved drawings and gives you insight to them and what is required and why to align with building standards and to achieve building sign-off.

Azure AI Services

Azure AI Services – For intelligent document analysis
Azure AI Search – For intelligent document search
Azure OpenAI – For generating responses and analysis
Azure Computer Vision – For analyzing construction drawings

Solution Architecture

Solution Azure AI Capabilities

AI Document Search

The Document Search component demonstrates effective integration of Azure AI services:

Azure AI Search

Indexes and semantically searches construction reference documents
Handles complex natural language queries about building standards
Retrieves the most relevant documents based on user queries
Returns confidence-ranked document matches with summaries

Azure OpenAI

Processes selected documents as context for generating responses
Synthesizes information across multiple reference sources
Follows specialized system instructions for construction knowledge
Generates comprehensive, well-formatted recommendations

Multi-stage AI Processing

Search Phase: Semantic search identifies relevant documents
Selection Phase: User reviews and selects contextual documents
Generation Phase: OpenAI creates recommendations using selected documents
Presentation: Results formatted with markdown for readability

This implementation demonstrates Azure’s document intelligence capabilities where AI Search finds relevant information that OpenAI then processes to create actionable construction guidance.

AI Drawing Analyzer

The Drawing Analyzer component demonstrates sophisticated Azure AI service integration:

Azure Computer Vision

Processes multiple file formats (JPG, PNG, PDF, TIFF)
Generates descriptive captions of construction drawings
Creates confidence-scored tags identifying key drawing elements
Analyzes multi-page PDFs page-by-page

Azure OpenAI

Powers the “Deep Analysis” functionality
Takes Computer Vision output as contextual input
Performs detailed construction-specific interpretation
Returns rich markdown-formatted technical insights

Progressive AI Analysis Flow

First-level: Basic visual understanding via Computer Vision
Second-level: Detailed construction interpretation via OpenAI
Presentation: Results with confidence scores and formatted highlights

This demonstrates effective AI service chaining where output from one Azure service becomes input for another, creating a comprehensive drawing analysis system.

Here are the reference standards and guidelines I used for New South Wales in Australia that contain all the information for compliance, health & safety.
For reference they are also in the Azure AI Search ‘guidelines’ subfolder.

Here is a screenshot of the structural drawings for the footings, walls and roof of my building project that was uploaded in the demo and analysed with Azure AI Vision and Azure Open AI.

For the demo see the Devpost Hackathon submission here.

Source Code on GitHub here.

Darren Robinson

Bespoke learnings from a Microsoft Identity and Access Management Architect using lots of Microsoft Identity Manager, Azure Active Directory, PowerShell, SailPoint IdentityNow and Lithnet products and services.