Azure AI Developer Hackathon
Posted on by Darren Robinson

Azure AI Developer Hackathon

I’ve just completed participating in the Azure AI Developer Hackathon that was looking to provide participants an opportunity to gain hands-on experience building intelligent applications on the Azure platform, supercharged by the power of GitHub Copilot.  

My submission is an Owner Builder AI Assistant.

Inspiration:

The inspiration for the Owner Builder AI Assistant stemmed directly from my own experience as a first-time owner builder. I’m currently embarking on the journey of a building project, and the reality of managing the project myself quickly became apparent. One of the most time-consuming and often frustrating aspects has been the process of requesting quotes from various trades. Ensuring these quotes are accurate, comprehensive, and aligned with building standards feels like a constant uphill battle, especially with limited prior experience. I envisioned a tool that could act as a digital assistant, guiding me through these complexities and empowering me to manage my construction project with greater confidence and efficiency.

The Owner Builder AI Assistant assists by generating prompts for AI based on your project phase and tasks that are then used in conjunction with building standards to generate either how to do the job yourself or what to ask for from a contractor. It then allows you to upload your approved drawings and gives you insight to them and what is required and why to align with building standards and to achieve building sign-off.

Azure AI Services

  • Azure AI Services – For intelligent document analysis
  • Azure AI Search – For intelligent document search
  • Azure OpenAI – For generating responses and analysis
  • Azure Computer Vision – For analyzing construction drawings

Solution Architecture

Solution Azure AI Capabilities

AI Document Search

The Document Search component demonstrates effective integration of Azure AI services:

Azure AI Search

  • Indexes and semantically searches construction reference documents
  • Handles complex natural language queries about building standards
  • Retrieves the most relevant documents based on user queries
  • Returns confidence-ranked document matches with summaries

Azure OpenAI

  • Processes selected documents as context for generating responses
  • Synthesizes information across multiple reference sources
  • Follows specialized system instructions for construction knowledge
  • Generates comprehensive, well-formatted recommendations

Multi-stage AI Processing

  1. Search Phase: Semantic search identifies relevant documents
  2. Selection Phase: User reviews and selects contextual documents
  3. Generation Phase: OpenAI creates recommendations using selected documents
  4. Presentation: Results formatted with markdown for readability

This implementation demonstrates Azure’s document intelligence capabilities where AI Search finds relevant information that OpenAI then processes to create actionable construction guidance.

AI Drawing Analyzer

The Drawing Analyzer component demonstrates sophisticated Azure AI service integration:

Azure Computer Vision

  • Processes multiple file formats (JPG, PNG, PDF, TIFF)
  • Generates descriptive captions of construction drawings
  • Creates confidence-scored tags identifying key drawing elements
  • Analyzes multi-page PDFs page-by-page

Azure OpenAI

  • Powers the “Deep Analysis” functionality
  • Takes Computer Vision output as contextual input
  • Performs detailed construction-specific interpretation
  • Returns rich markdown-formatted technical insights

Progressive AI Analysis Flow

  1. First-level: Basic visual understanding via Computer Vision
  2. Second-level: Detailed construction interpretation via OpenAI
  3. Presentation: Results with confidence scores and formatted highlights

This demonstrates effective AI service chaining where output from one Azure service becomes input for another, creating a comprehensive drawing analysis system.

Here are the reference standards and guidelines I used for New South Wales in Australia that contain all the information for compliance, health & safety.
For reference they are also in the Azure AI Search ‘guidelines’ subfolder.

Here is a screenshot of the structural drawings for the footings, walls and roof of my building project that was uploaded in the demo and analysed with Azure AI Vision and Azure Open AI.

For the demo see the Devpost Hackathon submission here.

Source Code on GitHub here.