Job Summary
Databricks Architect responsible for designing and implementing a scalable, cloud-native Lakehouse platform supporting RCIS Data Modernization. Lead the architecture of Bronze, Silver, and future Gold data layers, integrating Guidewire CDA, CIMax/DB2, legacy systems, and external regulatory data sources. Define metadata-driven ingestion, CDC frameworks, data quality, reconciliation, observability, governance, lineage, and security using Databricks, Delta Lake, Unity Catalog, AWS/Azure cloud services, and orchestration tools. Collaborate with enterprise architects, business stakeholders, data engineers, and governance teams to establish a trusted Single Source of Truth (SSOT), enabling audit-ready reporting, regulatory compliance, self-service analytics, and future AI-driven capabilities.
Key Responsibilities
Own end-to-end Databricks Lakehouse architecture and provide technical leadership across data ingestion, transformation, governance, reporting, and migration workstreams. Design canonical Policy, Claims, and Billing data models within the Silver Layer, ensuring harmonization between Guidewire and legacy systems. Establish metadata-driven ETL/ELT frameworks, data quality controls, reconciliation processes, lineage, observability, and security standards. Optimize Databricks clusters, Delta Lake performance, cost efficiency, and operational reliability. Govern Unity Catalog implementation, access controls, and compliance requirements. Support architecture reviews, SIT/UAT, cutover planning, production deployment, and hypercare activities. Mentor engineering teams and drive adoption of modern DataOps, automation, and AI-enabled accelerators.
Skill Requirements
Top 6 Key Skillsets – Databricks Architect (RCIS Data Modernization)Databricks Lakehouse ArchitectureDatabricks, Delta Lake, Unity Catalog, Delta Live Tables (DLT), Spark ArchitectureEnterprise Data Modeling & ModernizationCanonical Data Models, SSOT Design, Data Modernization, Bronze/Silver/Gold Layer ArchitectureData Engineering & IntegrationETL/ELT Frameworks, CDC, Data Ingestion, PySpark, Spark SQL, Real-Time & Batch ProcessingData Governance & QualityData Lineage, Metadata Management, Data Quality, Reconciliation, Observability, Audit ComplianceCloud & Platform EngineeringAzure Databricks, Azure Data Factory, Snowflake, Azure DevOps, CI/CD, Infrastructure as CodeCrop Insurance & Guidewire Domain ExpertiseGuidewire CDA, CIMax, Policy, Claims, Billing, APH Yield, Acreage, Premium, Subsidy, Indemnity, RMA Reporting