Sukanya Bhattacharjee

Learning assisted Interactive Modelling with Rough Freehand 3D Sketch Strokes

Paper Link Coming Soon

Accepted in 3DV 2025, Singapore

Freehand 3D sketches are a great medium to ideate and create visual content. However, generating 3D models from such rough sketches remains an unsolved, non-trivial task. We present an end-to-end interactive framework for rapid, incremental modelling from sparse, irregular 3D sketches. At the core of our solution, is sketchTransformer, a two-staged transformer network architecture, that fits parametric surface patches to a set of sketch strokes. We devise a novel pseudo height field representation that enables the sketchTransformer to handle noise and sparseness in the input strokes. Our method interactively evolves the surface model while maintaining smooth joins between nearby patches. We implement two frontends for our framework, one on the desktop and another as a mobile AR application, to illustrate how our method complements a standard 3D modelling pipelines. Our framework robustly handles a large variety of input $3$D strokes that competing methods cannot parse adequately.

Learning Assisted Interactive 3D modelling from 3D sketches

SIGGRAPH Asia 2023 Doctoral Consortium, Sydney NSW, Australia

Sketching is one of the most natural ways for representing any object pictorially. With the advent of Virtual Reality (VR) and Augmented Reality (AR) technologies, 3D sketching has become more accessible. But it is challenging to convert these sketches to 3D models in a manner consistent with the intent of the artist. Learning based methods provide an effective alternate paradigm for solving this classical problem of sketch based modelling in interactive platforms. Surface patches, inferred from 3D sketch strokes, can be treated as a primitive which are assembled to create complex object models. We present our proposed framework for this problem and discuss our solution to one of its core challenges. Our deep neural network based method allows users to create surfaces from a stream of sparse 3D sketch strokes. We also show integration of our method into an existing Blender based 3D content creation pipeline. This serves as a basis to solve a more complex problem of creating complete 3D object models from sparse 3D sketch strokes.

Deep Interactive Surface Creation from 3D Sketch Strokes

IJCAI 2022- AI for creativity track, Vienna, Austria

We present a deep neural framework that allows users to create surfaces from a stream of sparse 3D sketch strokes. Our network consists of a global surface estimation module followed by a local surface refinement. This facilitates in the incremental prediction of surfaces. Thus, our proposed method works with 3D sketch strokes and estimate a surface interactively in real time. We compare the proposed method with various state-of-the-art methods and show its efficacy for surface fitting. Further, we integrate our method into an existing Blender based 3D content creation pipeline to show its usefulness in 3D modeling.

State of Art on Sketch Based Content Creation

EUROGRAPHICS 2020, Sweden

Sketching is one of the most natural ways for representing any object pictorially. It is however, challenging to convert sketches to 3D content that is suitable for various applications like movies, games and computer aided design. With the advent of more accessible Virtual Reality (VR) and Augmented Reality (AR) technologies, sketching can potentially become a more powerful yet easy-to-use modality for content creation. In this state-of-the-art report, we aim to present a comprehensive overview of techniques related to sketch based content creation, both on the desktop and in VR/AR. We discuss various basic concepts related to static and dynamic content creation using sketches. We provide a structured review of various aspects of content creation including model generation, coloring and texturing, and finally animation. We try to motivate the advantages that VR/AR based sketching techniques and systems can offer into making sketch based content creation a more accessible and powerful mode of expression. We also discuss and highlight various unsolved challenges that current sketch based techniques face with the goal of encouraging future research in the domain.

Designing and evaluating hybrid games with tangible-virtual elements

We present our game, Cube2Pipes, as an attempt to investigate a unique gameplay design where we use a tangible 3D spatial puzzle, in the form of a 2X2 Rubik's Cube, as an interface to a tabletop mobile augmented reality (AR) game. The game interface adapts to user movement and interaction with both virtual and tangible elements via computer vision based tracking. This game can be seen as an instance of generic interactive hybrid systems as it involves interaction with both virtual and real, tangible elements. We present a thorough user evaluation about various aspects of the gameplay in order to answer the question as to whether hybrid gameplay involving both real and virtual interfaces and elements is more captivating and preferred by users, than standard (baseline) gameplay with only virtual elements. We use multiple industry standard user study questionnaires to try and answer this question. We also try to determine whether the game facilitates understanding of the spatial moves required to solve a Rubik's Cube, and the efficacy of a tangible puzzle interface to a tabletop AR game.

Character Animation in AR : A Mobile App Development Study

International Heritage Symposium and Exhibition (IHSE) 2020, National Museum, New Delhi, India

Digital preservation of the cultural heritages is one of the major applications of various computer graphics and vision algorithms. The advancement in the AR/VR technologies is giving the cultural heritage preservation an interesting spin due to its immense visualization ability. The use of these technologies to digitally recreate heritage sites and art is becoming a popular trend. A project, called Indian Digital Heritage (IDH), for recreating the heritage site of Hampi, Karnataka during Vijaynagara empire (1336 - 1646 CE) has been initiated by the Department of Science and Technology (DST) few years back. Immense work on surveying the site, collecting geographical and historic information about the life in Hampi, creating 3D models for buildings and people of Hampi and many other related tasks has been undertaken by various participants of this project. A major part of this project is to make tourists visiting Hampi visualize the life of people in ancient Hampi through any handy device. With such a requirement, the mobile AR based platform becomes a natural choice for developing any application for this purpose. We contributed to the project by developing an AR based mobile application to recreate a scene from Virupaksha Bazaar of Hampi with two components - author scene with augmented virtual contents at any scale, and visualize the same scene by reaching the physical location of augmentation. We develop an interactive application for the purpose of digitally recreating ancient Hampi. Though the focus of this work is not creating any static or dynamic content from scratch, it shows an interesting application of the content created in a real world scenario.

Vertex Fault-Tolerant Geometric Spanners for Weighted Points

International Journal of Computational Geometry and Applications 2022

Given a set 𝑆 of 𝑛 points, a weight function 𝑤 to associate a non-negative weight to each point in 𝑆, a positive integer 𝑘≥1 , and a real number 𝜖>0, we present algorithms for computing a spanner network 𝐺(𝑆,𝐸) for the metric space (𝑆,𝑑𝑤) induced by the weighted points in 𝑆. The weighted distance function 𝑑𝑤 on the set 𝑆 of points is defined as follows: for any 𝑝,𝑞∈𝑆, 𝑑𝑤(𝑝,𝑞) is equal to 𝑤(𝑝)+𝑑𝜋(𝑝,𝑞)+𝑤(𝑞) if 𝑝≠𝑞, otherwise, 𝑑𝑤(𝑝,𝑞) is 0. Here, 𝑑𝜋(𝑝,𝑞) is the Euclidean distance between 𝑝 and 𝑞 if points in 𝑆 are in ℝ𝑑, otherwise, it is the geodesic (Euclidean) distance between 𝑝 and 𝑞. The following are our results: (1) When the weighted points in 𝑆 are located in ℝ𝑑, we compute a 𝑘-vertex fault-tolerant (4+𝜖)-spanner network of size 𝑂(𝑘𝑛𝜖2). (2) When the weighted points in 𝑆 are located in the relative interior of the free space of a polygonal domain 𝒫, we detail an algorithm to compute a 𝑘-vertex fault-tolerant (4+𝜖)-spanner network with 𝑂(𝑘𝑛ℎ+1√𝜖2lg𝑛) edges. Here, ℎ is the number of simple polygonal holes in 𝒫. (3) When the weighted points in 𝑆 are located on a polyhedral terrain 𝒯, we propose an algorithm to compute a 𝑘-vertex fault-tolerant (4+𝜖)-spanner network, and the number of edges in this network is 𝑂(𝑘𝑛𝜖2lg𝑛).

Geodesic additive weighted fault-tolerant spanners

COCOONS 2019

Let S be a set of n points and let w be a function that assigns non-negative weights to points in S. The additive weighted distance 𝑑𝑤(𝑝,𝑞) between two points 𝑝,𝑞∈𝑆 is defined as 𝑤(𝑝)+𝑑(𝑝,𝑞)+𝑤(𝑞) if 𝑝≠𝑞 and it is zero if 𝑝=𝑞. Here, d(p, q) is the geodesic Euclidean distance between p and q. For a real number 𝑡>1, a graph G(S, E) is called a t-spanner for the weighted set S of points if for any two points p and q in S the distance between p and q in graph G is at most t.𝑑𝑤(𝑝,𝑞) for a real number 𝑡>1. For some integer 𝑘≥1, a t-spanner G for the set S is a (k, t)-vertex fault-tolerant additive weighted spanner, denoted with (k, t)-VFTAWS, if for any set 𝑆′⊂𝑆 with cardinality at most k, the graph 𝐺∖𝑆′ is a t-spanner for the points in 𝑆∖𝑆′. For any given real number 𝜖>0, we present algorithms to compute a (𝑘,4+𝜖)-VFTAWS for the metric space (𝑆,𝑑𝑤) resulting from the following: (i) points in S are in the free space of the polygonal domain, (ii) points in S lying on a terrain.

Fault-tolerant additive weighted geometric spanners

CALDAM 2019

Given a set S of n points, a weight function w to associate a non-negative weight to each point in S, a positive integer k > 1, and a real number 𝜖 > 0, we present algorithms for computing a spanner network G(S, E) for the metric space (S, 𝑑𝑤) induced by the weighted points in S. The weighted distance function 𝑑𝑤 on the set S of points is defined as follows: for any p, q ∈ S, 𝑑𝑤(p, q) is equal to w(p) + dpi(p, q) + w(q) if p != q, otherwise, 𝑑𝑤(p, q) is 0. Here, dpi(p, q) is the Euclidean distance between p and q if points in S are in R^^d, otherwise, it is the geodesic (Euclidean) distance between p and q. The following are our results: (1) When the weighted points in S are located in R^d, we compute a k-vertex fault-tolerant (4+𝜖 )-spanner network of size O(k n). (2) When the weighted points in S are located in the relative interior of the free space of a polygonal domain P, we detail an algorithm to compute a k-vertex fault-tolerant (4+𝜖 )-spanner network with O(kn.(h+1)^2/\𝜖 ^2 lg(n)) edges. Here, h is the number of simple polygonal holes in P. (3) When the weighted points in S are located on a polyhedral terrain T, we propose an algorithm to compute a k-vertex fault-tolerant (4+𝜖 )-spanner network, and the number of edges in this network is O(kn/𝜖 ^2 lg(n)).

Content Based Image Retrieval using color, texture and shape features

BTP

Due to the explosive growth of digital imaging on the World Wide Web, efficient and effective retrieval techniques of images are required. The traditional manual indexing is more time consuming due to its pixel to pixel comparison for each image. Problems with traditional methods of image indexing have led to the rise of interest in techniques for retrieving images on the basis of automatically derived features as color, texture and shape. Content based image retrieval is one such technique that retrieves the desired images from a large collection on the basis of features such as color, texture and shape that can be automatically extracted from the images themselves. Content-Based Image retrieval (CBIR) is a promising approach because of its automatic indexing and retrieval based on their semantic features and visual appearance. The similarity of images depends on their feature representation and feature dissimilarity function. Thus, CBIR provides an efficient and accurate way to solve this issue. Currently available methods of CBIR do not include all the features of an image, usually isolating one. So, we have developed a technique for CBIR which includes all the three features and the comparative analysis shows that this model is efficient.

Central Instrument Facility (CIF) Booking Portal

Internship

Designed an online portal for booking instruments in CIF to avoid manual processing. Digitizing the book-keeping process of instrument usage and allocation.